5\. DATA COLLECTION

Nearly 300 interviewers collected data for the 2001 National Survey of
Veterans (NSV 2001) over a 9-month period. This chapter describes the
computer system we used to collect the data, the procedures we employed
to ensure successful interviews with respondents, and the special
operations we instituted to enhance response rates.

# 5.1 The CATI System

####  Questionnaire Administration

Westat's proprietary Cheshire computer-assisted telephone interviewing
(CATI) system is designed to handle very large and complex survey
instruments. Not only does this system greatly enhance data quality by
automating most data collection tasks, it also reduces respondent burden
by streamlining the questions. The CATI system is particularly
appropriate for the NSV 2001, with its complicated questionnaire, List
and RDD Sample types, and household screening requirements for RDD
Sample cases. By using the CATI system, we ensured that the NSV 2001
instrument was administered correctly, the samples were managed
efficiently, and data were recorded properly. For example, the software
guided interviewers through the programmed skip patterns; performed
range, logic, and consistency checks; determined when cases should be
called and how often; kept track of which cases belonged to each sample;
and stored all data to a common database. Detailed descriptions of these
and other CATI system capabilities follow.

**Display.** The standardized display of survey questions, interviewer
instructions, and response options contributed to better data quality by
helping to ensure that each interviewer administered the instrument in
the same manner. The program also displayed all relevant respondent
information, automatically inserting into question text personalized
data such as the respondent's name.

**Sample Management.** The CATI system differentiated between the RDD
and List Samples, and provided the appropriate questions for the
interviewer to ask. RDD Sample respondents automatically received the
household screening questionnaire (see Appendix A), while List Sample
members were administered a verification question (see Appendix B). The
program implemented decision rules based on military service questions
in the screening questionnaire and the extended questionnaire to
determine eligibility. It then selected eligible persons for an
interview and prompted interviewers to terminate the interview for
ineligible respondents. The CATI system also was able to evaluate the
samples separately in reference to their target production goals.

**Online Data Entry.** Data were captured instantly in the CATI system
when interviewers typed respondent answers as they conducted the
interview. The program offered a variety of ways to correct keying
errors. Interviewers corrected data entry mistakes immediately by
backing up through the CATI screens, reentering information, then moving
forward again through the corrected question path. For mistakes
discovered after the interview was over, or those too time consuming to
correct during the interview, interviewers entered online comments for
review by the database manager, or submitted a hard copy problem sheet
to their shift supervisor. (See Chapter 7 for a more thorough discussion
of database management.)

**Skip Patterns.** The NSV 2001 questionnaire contained numerous, often
complicated, skip patterns. Once the questionnaire was programmed into
the CATI system, however, these skips were administered automatically
and consistently. As the interviewer entered a respondent's answers, the
program determined and then displayed the correct question path. The
CATI system accessed the sample file or other data collected from the
respondent and presented specific sections of the instrument or
different versions of the questions in accordance with programming
instructions. The software also used existing data to determine whether
or not to administer certain questions. For example, if the veteran
provided exact dates of induction and release from active duty in the
Military Background section of the main interview, the CATI system
calculated and automatically entered that veteran's periods of service.
This feature reduced response burden for some veterans, which decreased
overall survey administration time.

**Range Checks.** The CATI system is capable of performing both hard and
soft range checks. Hard range checks set the upper and lower limits of
allowable item responses. Hard ranges were required for all variables. A
question with response categories of 1 through 12 accepted only those
values, in addition to the required missing values of "Don't Know" and
"Refused." Exhibit 5-1 is an example of a hard range check. Soft range
checks queried implausible responses but allowed the entry to stand if
unchanged after being probed by the interviewer. We used soft ranges for
questions about dates or quantitative information. Exhibit 5-2 is an
example of a soft range check.

**Logic and Consistency Checks.** Logic checks ensured that data
collected at different points in the interview were internally
consistent. If responses were inconsistent, interviewers were prompted
through a series of custom screens to verify the recorded data. The
program accepted corrected data when obtained. If the responses remained
inconsistent, the case was flagged for review by the database manager
and the interview continued. Exhibit 5-3 is an example of a logic check.

Exhibit 5-1. Example of CATI specifications with hard range

**PROGRAMMER NOTE 1:**

**IN ALL MONTH FIELDS, HARD RANGE = 1 -- 12.**

**IN ALL INSTANCES WHERE OTHER SPECIFY = YES PROVIDE 30 CHARACTER OTHER
SPECIFY FIELD.**

**IN MB0a, RANGE FOR YEAR = 1885 -- (1983).**

**VETS.DOBMM, VETS.DOBYYYY**

MB0a. First, I'd like to ask you for the month and year you were born.

> \|\_\_\|\_\_\| MONTH
>
> \|\_\_\|\_\_\|\_\_\|\_\_\| YEAR
>
> REFUSED -7
>
> DON'T KNOW -8
>
> 1\. JANUARY 7. JULY
>
> 2\. FEBRUARY 8. AUGUST
>
> 3\. MARCH 9. SEPTEMBER
>
> 4\. APRIL 10. OCTOBER
>
> 5\. MAY 11. NOVEMBER
>
> 6\. JUNE 12. DECEMBER

Exhibit 5-2. Example of CATI specifications with soft range

**PROGRAMMER NOTE 63:**

**IN (SD13) DEPEND HARD RANGE = 0 -- 15. SOFT RANGE = 0 -- 10.**

\[O6a.\]

**VETS.DEPEND**

SD13. During the year 2000, how many children depended on you for at
least half of their support?

> NUMBER \|\_\_\|\_\_\|
>
> REFUSED -7
>
> DON'T KNOW -8

Exhibit 5-3. Example of CATI logic check

> 1.1501 MB15A 100010300101 - (410) 555-7834 - 16:13
>
> I have recorded that you were released from active duty on
>
> January 5, 1934. That date is earlier than the
>
> date you began active duty. Please tell me the date you
>
> [began]{.underline} your active duty.
>
> (5 ) (5 ) (1931 )
>
> MONTH DAY YEAR
>
> 1\. JANUARY 7. JULY
>
> 2\. FEBRUARY 8. AUGUST
>
> 3\. MARCH 9. SEPTEMBER
>
> 4\. APRIL 10. OCTOBER
>
> 5\. MAY 11. NOVEMBER
>
> 6\. JUNE 12. DECEMBER

1.1502 MB15B 100010300101 - (410) 555-7834 - 16:13

And again, what was the date you were [released]{.underline} from active
duty?

(1 ) (5 ) (1945 )

MONTH DAY YEAR

1\. JANUARY 7. JULY

2\. FEBRUARY 8. AUGUST

3\. MARCH 9. SEPTEMBER

4\. APRIL 10. OCTOBER

5\. MAY 11. NOVEMBER

6\. JUNE 12. DECEMBER

**Data Storage.** The CATI system stored all administrative and
substantive data collected during each interview administration. One
segment of the software recorded each attempted contact with every
respondent. It stored the date and time of the call attempt, interviewer
identification, work class, disposition code, and termination
information. Another segment recorded every keystroke made during every
interview. If an interview was interrupted and data lost due to computer
failure, this file was used to restore the original responses. The
system also stored all hard range failures and soft range overrides.

####  CATI Case Management and Call Scheduling

Telephone numbers loaded into the CATI system became available to
interviewers through the CATI scheduler. An autodialer dialed the
numbers, reducing interviewer dialing time as well as eliminating the
possibility of dialing a telephone number incorrectly. The CATI
scheduler kept track of the number of calls made to each telephone
number and automatically closed out those that had reached the maximum
number of contact attempts without completing an interview. The CATI
scheduler also ensured that cases were called at the appropriate times,
using rules developed to minimize the number of calls to any given
household and to reduce nonresponse. For example, the week was divided
into day and time period categories through which the system moved each
case in a specified pattern of call attempts. If the first call attempt
was in the evening and resulted in no answer, the CATI scheduler
automatically set the next call attempt for another time of day and a
different day of the week. For cases where the interviewer made contact
and scheduled an appointment to complete the screener, or to begin or
continue the extended interview, the system held the case then released
it to the next available interviewer at the scheduled day and time.
Screener interviews that were interrupted were restarted at the
beginning of the screening questionnaire when the household was reached
again. Interrupted extended interviews began again at the beginning of
the questionnaire section from which the case had exited. Cases with
scheduled appointments were given priority over all other cases.

Another function of the CATI scheduler was to manage the flow and
assignment of cases to interviewers. The scheduling system analyzed a
number of factors when determining what case to assign to a particular
interviewer, including interviewer skills required, call history, time
zone of the respondent, day of the week, time of day, and priority
weighting of cases by sample group. This analysis required no indepth
knowledge on the part of the interviewer, and it maximized the number of
completed interviews while minimizing the number of nonproductive
interviewer hours. The CATI scheduling system also appropriately
assigned to specially trained interviewers those cases that needed to be
traced, refusal cases, language problems, and cases requiring proxy
interviews. Finally, the system maintained a log of all call attempts
and their results.

The CATI scheduler was customized to accommodate the two NSV 2001
samples in a variety of ways. First, we programmed the CATI system to
track multiple telephone numbers for each List Sample veteran. When an
interviewer learned of a new telephone number for a veteran, it was
entered directly into the system and dialed at the next call attempt.
Project staff or the database manager entered telephone numbers
collected offline from sources other than household members (such as the
VA and directory assistance). Multiple telephone numbers were also
tracked for RDD veterans (for example, veterans who had moved out of the
household they were living in at the time the screener survey was
completed).

We further programmed the CATI scheduler to optimize when and how often
a case was called based on sample type. Calls to List Sample members
were made primarily in the evenings and on weekends because we were much
more likely to contact a household resident during those hours. Initial
calls to the RDD Sample were made primarily during the day. While this
did not optimize household contact, interviewers were able to quickly
close out the nonworking and business numbers that remained even after
the sample had been prescreened for them.

####  Programming and Testing the CATI System

The CATI development staff was led by a systems analyst who was
responsible for coordinating all survey programming and testing
activities. The systems analyst worked closely with the questionnaire
designers to ensure that the CATI program specifications addressed every
aspect of the NSV 2001 instrument. The specifications included variable
name and location, valid response categories, and instructions for item
nonresponse. The programming staff then used those specifications to
program the instrument, verifying that the resulting instrument
conformed to the specifications and ensuring that no errors were hidden
in the specifications.

Westat thoroughly tested all features of the system, including those
associated with scheduling, interviewing, data storage, and editing.
After each section was programmed, it went through multiple rounds of
intensive internal testing. In the first round, the programmers used the
specifications to proof the screens, check the data dictionary, and
match the programmed language flow against the survey instrument's skip
patterns. The next round of testing focused on transitions from one
questionnaire section to another. We also tested restart points to
ensure that the flow between sections and topics was smooth, and that
questionnaire sections appeared in the proper sequence.

For the final round of testing, result codes, randomization routines,
standard function keys, database fields, flags, array variables, timing
variables, audit trails, and database structure were validated. We
proofed all text on the screens a final time and confirmed the accurate
collection of data into the survey database. Testing scenarios also
targeted the questionnaire delivery system (the CATI scheduler). The
focus here was on the proper delivery of cases to interviewers, whether
appointments were correctly made and kept, and how interview break-offs
were treated.

Just as during the data collection period, forms were filled out
whenever a potential problem was found. These problem sheets described
the situation, listed key variables, indicated expected results, and
documented the actual result. The database manager logged each problem
sheet and routed it to the appropriate person. Once the problem was
resolved and the changes were implemented in the testing environment,
the database manager sent the problem sheet back to the originator so
that the test could be rerun. This process ensured that at least two
people verified that a problem had been resolved. Westat kept its CATI
program and systems documentation, test plans, and test sets up to date,
thus ensuring that changes to the instrument could be easily
incorporated and tested. CATI system version control software maintained
a history of changes.

# 5.2 Interviewing Operations

Data collection for the NSV 2001 began on February 12, 2001 and ended
November 12, 2001. Interviewing was conducted from six of Westat's
Telephone Research Center (TRC) facilities. The telephone centers
operated seven days a week.

####  Management, Staffing, and Scheduling

The telephone operations manager coordinated all training and
interviewing activities with supervisory staff in the six TRCs. For
training, the operations manager prepared the agenda, produced the
manuals, scripts, and interviewer forms, and created exercises and role
plays. The operations manager also developed procedures for special
operations such as refusal conversion, proxy interviewing, and tracing.
During data collection, the operations manager reviewed and resolved
problem cases. To help evaluate the progress of data collection, the
operations manager produced and presented to the project team a weekly
summary of the status of all cases by screener and main interview, by
interim and final results, and by sample type. Finally, the operations
manager oversaw all interviewer scheduling and staffing.

Interviewers made most calls in the evenings and on weekends, when
people were most likely to be home. We assigned primary responsibility
for handling language problem cases to experienced Spanish-speaking
interviewers. A case was coded as a language problem if an interviewer
was unable to communicate with the respondent in English.
Spanish-speaking interviewers then recontacted the case to determine if
an English-speaking person lived in the household. We trained
interviewers who had clear, deep voices to follow up with hearing
problem cases. We chose skilled interviewers at all six TRC facilities
to conduct refusal conversion interviewing. We selected interviewers at
one location to perform the majority of tracing calls, and updated the
CATI system with the List Sample telephone numbers obtained through
those efforts.

####  Interviewer Monitoring

Supervisors regularly monitored interviewer performance. Because most
refusals occur at the point of contact, supervisors paid particular
attention to the initial contact. Monitoring sessions lasted a minimum
of ten minutes, during which supervisors made notes of interviewers'
strengths and weaknesses on a monitoring form. Supervisors discussed the
results with each interviewer immediately following a monitoring
session. They provided positive feedback along with pointers for
improvement in areas such as gaining cooperation and probing.
Supervisors also used a weekly interviewer productivity report to
identify candidates for training on more difficult tasks such as refusal
conversion, as well as interviewers who needed additional training.

####  Confidentiality

All Westat personnel, including interviewers and professional staff,
signed a statement that they would maintain the confidentiality of all
survey data. During data collection, interviewers assured each
respondent that his or her answers were protected under the Privacy Act
(see Exhibit 5-8) and informed respondents that a copy of the Privacy
Act statement was available upon request (see "Providing a Privacy Act
Letter" later in this chapter).

# 5.3 Special Operations

In addition to the standard data collection operations described in the
previous sections, Westat employed a variety of strategies unique to the
NSV 2001 for achieving the maximum possible response rates. These
special operations included contact information update measures, proxy
interviewing, and response rate enhancement measures such as refusal
conversion and data retrieval.

####  Contact Information Update Measures

####  *Cleaning the List Sample and Matching Addresses*

Before loading the List Sample contact information into our CATI system,
we processed the file through a series of automated and manual cleaning
procedures that standardized the appearance of all addresses. Once the
sample was cleaned, we matched cases with address information against
the National Change of Address Registry to ensure that we had the most
recent address. We also matched all cases with address and/or telephone
information against the databases of two address matching vendors. This
step allowed us to either obtain a telephone number for those cases with
no telephone numbers or to update existing telephone numbers. Those
veterans for whom we had a telephone number but no address were loaded
into the CATI system without mailing an advance letter. Those cases for
which we had a Social Security number and an address, but no telephone
number, were sent to a credit bureau for telephone information searching
by Social Security number. Those cases for which we had an address but
no telephone number were sent an advance letter, which asked for updated
contact information, including a telephone number. These cases, along
with those for which we had neither a telephone number nor address, were
sent immediately to our tracing operation. During the cleaning process,
veterans identified as institutionalized were coded as out of scope so
that no further contact attempts would be made.

####  *Sending an Advance Mailing*

After List Sample cleaning and address matching, we sent each List
Sample veteran an advance mailing that included a letter from the
Secretary of the Department of Veterans Affairs and a letter from the
Westat project director (see Exhibits 5-4 and 5-5). The letters informed
veterans about the study and the importance of their participation. We
included an address update form and postage-paid envelope so that
veterans could update their names, addresses, or telephone numbers by
mail. Veterans were also given the option to call the NSV 2001 dedicated
toll-free number to provide their current address and telephone
information. To help us identify institutionalized veterans, the address
update form also asked veterans to indicate whether the address was for
a private residence, hospital, assisted living facility, retirement
home, or nursing home. (As discussed in Chapter 3, we did not interview
List Sample institutionalized veterans.)

We mailed 13,010 advance letters in two waves. In response to the
advance mailings, we received 4,061 address update forms through the
mail. Of those, however, less than half contained contact information
updates. Those with no update did indicate whether their address was for
a private residence or some other type of residence. We reviewed all
additional information we received about a veteran, then entered it into
the CATI Update system where it became effective immediately. Table 5-1
shows the type of information received on the address update form.

Table 5-1. List Sample advance mailout responses

  -------------------------------------------------- --------------------
  Address update                                     825

  Telephone number update                            917

  Name correction                                    48

  Deceased                                           197

  Not a veteran                                      1

  Form returned with no update                       2,073

  Total responses received                           4,061
  -------------------------------------------------- --------------------

Exhibit 5-4. VA advance letter

![](media/image1.png){width="6.485416666666667in"
height="8.749305555555555in"}

Exhibit 5-5. Westat advance letter

WESTAT

National Survey of Veterans

1650 Research Blvd. Rockville, MD 20850-3129 301 251-1500 FAX 301
294-2040

> \<\<DATE\>\>

\<\<BASMID\>\>

\<\<FULLNAME\>\>

\<\<ADDR1\>\>

\<\<ADDR2\>\>

\<\<CITY\>\>, \<\<STATE\>\> \<\<ZIP\>\>

Dear \<\<FULL_LOWER\>\>:

The Department of Veterans Affairs (VA) has chosen Westat to conduct its
National Survey of Veterans. The enclosed letter from Anthony J.
Principi, Secretary of Veterans Affairs, explains that the VA is
conducting this survey to obtain information for planning benefits and
services for all veterans.

A Westat interviewer will call you in the next few weeks to conduct a
telephone interview. Your answers, and those of other participating
veterans, will give the VA an up-to-date picture of the whole U.S.
veteran population. If you wish to verify Westat's role in the survey,
please call the toll-free VA number mentioned in Secretary Principi's
letter: 1-800-827-1000.

While your participation is voluntary, you are one of only a few
thousand randomly selected veterans whose answers will represent all 25
million veterans. Your individual participation will have a significant
impact on the survey's outcome. We cannot interview another veteran in
your place. I want to assure you that the information you provide is
protected under the Privacy Act and section 5701 of Title 38 of the U.S.
Code. The VA will use the information you provide to evaluate current VA
policies, programs and services for veterans and in deciding how to help
veterans in the future.

Enclosed is a National Survey of Veterans Address Correction and Update
Form. Westat will call you at the phone number printed on this form. If
this phone number or address is incorrect, please provide the correct
information and mail the form to Westat in the postage-paid envelope
provided. If you prefer, you may call Westat toll-free at 1-888-258-2194
to provide a correct phone number and address. Ask to speak with the
Veterans Survey Manager.

Thank you. We greatly appreciate your help.

> Sincerely,
>
> ![](media/image2.png)
>
> John C. Helmick
>
> Project Director

####  *Conducting Tracing Operations*

Tracing is a process by which we attempt to find valid contact
information for veterans for whom we have no or incomplete information
in our records. For some List Sample veterans, we did not have a valid
telephone number, the existing number was nonworking, or we did not have
a telephone number at all. In an effort to contact these veterans, we
placed directory assistance calls. We pursued two leads per tracing
case. If the veteran was not listed at the address in our records, and
there were six or fewer listings with that veteran's last name in the
town or surrounding area, tracing interviewers requested the first two
telephone numbers. The interviewers called both households asking for
information about the veteran. When we identified new telephone numbers,
we entered them into the CATI system within 24 hours. Over the course of
data collection, tracing interviewers worked 5,350 cases.

####  *Contacting Cases With Unpublished Telephone Numbers*

During the tracing operation, we often encountered List Sample veterans
who had an address in our records but whose telephone number was not
published. We mailed these veterans a letter requesting their
participation in the study and providing a toll-free number for them to
contact us (see Exhibit 5-6). We mailed these letters to 555 veterans,
of whom 124 completed the extended interview. We also learned that 17 of
the 555 had died and 4 were institutionalized.

####  *Contacting Cases With No Telephone Numbers*

We had no telephone number and only a Post Office Box address for 65
List Sample veterans. Without a street address, we were unable to
process these cases through our tracing operation. Instead, we mailed a
letter to these veterans asking them to call the NSV 2001 toll-free
number for an interview (see Exhibit 5-7). Of the 65 veterans who
received this letter, 15 completed an extended interview. One was
reported deceased.

Exhibit 5-6. Letter to veterans with unpublished telephone numbers

WESTAT

National Survey of Veterans

1650 Research Blvd. Rockville, MD 20850-3129 301 251-1500 FAX 301
294-2040

> DATE

«BASMID»

«FULLNAME»

«ADDR1»

«CITY», «STATE» «ZIP»

Dear «FULL_LOWER»:

**We need your help.** The Department of Veterans Affairs (VA) has
chosen Westat to conduct its National Survey of Veterans. We would like
to interview you for the survey. Unfortunately, we have been unable to
reach you because your telephone number is unpublished.

The VA is conducting this survey to obtain information for planning
benefits and services for all veterans. While your participation is
voluntary, you are one of only a few thousand randomly selected veterans
whose answers will represent all 25 million veterans. Your individual
participation will have a significant impact on the survey's outcome. We
cannot interview another veteran in your place. Your answers, and those
of other participating veterans, will give the VA an up-to-date picture
of the whole U.S. veteran population.

I want to assure you that the information you provide is protected under
the Privacy Act and section 5701 of Title 38 of the U.S. Code. The VA
will use the information you provide to evaluate current VA policies,
programs and services for veterans and in deciding how to help veterans
in the future.

We understand your desire to keep your telephone number closely guarded.
For that reason, we invite you to call Westat toll-free at
1-888-258-2194 to begin an interview or schedule an appointment at a
time that is convenient for you. Please ask to speak with the Veterans
Survey Manager. Or, you may provide us with your telephone number on the
enclosed National Survey of Veterans Address Correction and Update Form
and mail it to Westat in the postage-paid envelope. Your telephone
number will only be used to contact you for completing the National
Survey of Veterans and will not be disclosed for any other purposes.

Thank you. We greatly appreciate your help.

> Sincerely,
>
> ![](media/image2.png)
>
> John C. Helmick
>
> Project Director

Exhibit 5-7. Letter to veterans with no telephone number

WESTAT

National Survey of Veterans

1650 Research Blvd. Rockville, MD 20850-3129 301 251-1500 FAX 301
294-2040

DATE

FNAME LNAME

ADDRESS

CITY, STATE ZIP

Dear Mr./Ms./Mrs. LNAME:

Thank you for your recent reply to our letter regarding the National
Survey of Veterans. Westat is conducting this study for the Department
of Veterans Affairs.

Based on your response, we understand that you are not able to receive a
telephone call in order to complete the survey. Since it is necessary
that all surveys be completed by telephone, and your responses are very
important to us, we would like to provide you an option that would make
it possible for you to participate in the survey.

To allow us the opportunity to complete a survey with you, we are asking
that you call our toll-free telephone number and ask to speak to one of
our telephone interviewers. Please tell our staff that you are calling
to complete the **National Survey of Veterans**. Once you have
identified the survey for which you are calling to participate, you will
need to give your survey identification number. This number is listed
below as the **ID Number**. Please try to call within the next 2 weeks
and during the hours outlined below.

Phone Number: **1-888 258-2194**

Project: **National Survey of Veterans**

ID Number: **BASMID**

Calling Hours:

> **Monday-Friday 9:00 am -- 10:00 pm (Eastern Daylight Time)**

**Saturday 10:00 am -- 4:00 pm (Eastern Daylight Time)**

**Sunday 2:00 pm -- 8:00 pm (Eastern Daylight Time)**

Thank you very much for helping us with this important study. We look
forward to hearing from you.

Sincerely,

![](media/image3.png){width="1.698611111111111in"
height="0.5659722222222222in"}

John C. Helmick, Jr.

Project Director

####  *Searching Credit Bureaus*

We sent the Social Security numbers of veterans for whom we had no
telephone number (or an invalid telephone number) to a credit bureau.
Toward the end of data collection, we performed the same operation on
all cases coded as nonlocatable and all cases in our tracing queue. Some
of these nonlocatable and tracing cases were likely the same ones that
had been processed through the credit bureau at the beginning of the
study. Most, however, were cases for which we had incomplete contact
information. The cases returned from the credit bureau with addresses
were then sent to our address matching vendors for telephone numbers.
Table 5-2 shows the results of the end-of-data collection credit bureau
search.

Table 5-2. Results of end-of-data collection credit bureau search

  --------------------------------------------------------- -------------
  Number of cases sent to the credit bureau                 3,197

  Number for which a telephone number was obtained          513

  Complete extended interviews                              150

  Deceased                                                  6

  Institutionalized                                         5
  --------------------------------------------------------- -------------

####  Proxy Interviewing

The NSV 2001 data collection plan permitted veterans with serious
health, cognitive, or communication problems to be represented in the
final sample, despite the fact that many of these veterans were unable
to respond for themselves or, at least, unable to respond to a telephone
interview. Veterans with such disabilities were:

Too mentally incapacitated to understand the questions or to remember or
formulate their responses;

Too physically incapacitated, too ill, or too weak to listen and respond
to the questions for the duration of the interview;

Too hearing-impaired to hear the interview questions; or

Too speech-impaired to communicate their responses to the interviewer.

We included the responses of veterans with these disabilities (referred
to as the sampled veterans) by allowing proxy respondents to participate
in the survey for them. Our estimates of the number of cases that would
require proxies, while large enough to confirm the need for a proxy
interview protocol, did not justify an entirely separate data collection
instrument and methodology. Instead, the proxy interview protocol used a
modified form of the existing CATI data collection instrument and
telephone methodology.

We established rules to help interviewers determine when a proxy
interview was appropriate, permitting them only under the four
circumstances described above. We did not interview proxy respondents
for any other reason, such as convenience, personal preference,
temporary unavailability of the veteran, or temporary illness from which
the veteran would recover before the end of the data collection period.
The most common way in which interviewers identified the need for a
proxy was when a friend or relative (and occasionally the veterans
themselves) reported that the sampled veteran was incapable of
participating in the study. In much rarer situations, the veteran was
willing to be interviewed, but during the course of the interview it
became apparent that he or she could not comprehend the questions or
remember any reasonable amount of information about himself or herself.

We also trained interviewers to identify a suitable proxy for veterans
who could not participate and to secure the telephone number needed to
call the proxy. Such a proxy needed to be familiar with the sampled
veteran's circumstances and possess some specific knowledge of his or
her health and medical history. In most cases, the person who informed
the interviewer of the veteran's inability to respond was the
appropriate proxy, and the proxy was often located at the same telephone
number. If the potential proxy was not knowledgeable enough about the
veteran or was unavailable, interviewers attempted to find another
proxy. If no proxy could be found or the suitable proxy refused to
respond, the final result of the case reflected the sampled veteran's
situation, not the outcome of contacting the proxy.

We trained a small group of interviewers to conduct proxy interviews. If
a proxy interviewer identified the need for a proxy in the course of
ordinary contact attempts, and the proxy was available, he or she
proceeded directly with the proxy interview. If, however, an interviewer
without proxy interview training identified such a need, an appointment
was made and the case was held in the CATI scheduler proxy work class
for a proxy interviewer. To accommodate the time needed to identify and
train proxy interviewers, we did not begin proxy interviewing until
April, 2001. All proxy cases from the beginning of data collection were
held in the proxy work class until that time, and proxy interviewing
continued through the end of data collection.

As noted earlier, we knew that relatively few veterans would require a
proxy interview. This fact, coupled with our desire for comparable data
from the proxy cases and the self-respondents, led us to modify the
existing questionnaire rather than create a separate proxy instrument.
We identified all questions in the NSV 2001 questionnaire that proxies
would likely be able to answer for the sampled veterans. We eliminated
some questions (such as the VSF12 items in the Health Background
section) that were too personal for a proxy to answer adequately.
Finally, we trained proxy interviewers to rephrase select questions in
the third person to remind proxies that while the questions were
directed at them, they were in fact about the sampled veteran. Rather
than program the CATI instrument to display all the third person
pronouns, we trained the proxy interviewers to use the third person as
they read the questions. We chose this cost effective measure because
few questions actually needed rephrasing, and any rephrasing usually
occurred within a pattern of repeating questions. Furthermore, the
interviewers doing the rephrasing, in addition to being highly skilled,
also had at least 2 months of experience administering the questionnaire
before proxy interviewing began.

We expected "Don't Know" responses to be more common among proxy
respondents than self-respondents. We found, however, that proxy
respondents were often very knowledgeable about the sampled veteran's
background information, military service, medical conditions, and
treatment. When proxies did have limited knowledge in certain areas,
interviewers encouraged them to provide the best response they could. In
a few cases, attempts to contact the proxy resulted in the discovery
that a sampled veteran could, after all, respond for himself or herself.
In these instances, we conducted the interview with the veteran, and the
case was correctly coded as a regular, nonproxy interview.

We identified 1,031 cases as needing a proxy respondent. We completed
interviews with 734 proxy respondents. Another 28 cases turned out to be
ineligible for the study. Of the cases initially identified as needing
proxies, an additional 45 interviews were eventually conducted with the
sampled veteran. Thus, only 3.7 percent of all interviews were conducted
with proxies.

####  Response Rate Enhancement Measures

####  *Establishing a Toll-free Telephone Number* 

We established a toll-free telephone number dedicated solely to the NSV
2001. Only supervisory staff answered this number. We included this
number on all mailout materials, and provided it to respondents who
wanted more information about the study before consenting to an
interview. Veterans with no telephone in the household called the
toll-free number to complete an interview (see the previous sections on
"Contacting Cases with Unpublished Telephone Numbers" and "Contacting
Cases with No Telephone Numbers"). Difficult-to-reach respondents often
called the toll-free line to participate, or at least tell us when they
would next be available for an interview.

Respondents who wished to contact the VA directly about the NSV 2001
were also provided with the VA's main toll-free number (800-827-1000).
This number offered an automated menu where veterans could obtain
information about various VA benefits, or remain on the line for the
next available counselor. Because information about the NSV 2001 was not
part of the automated menu, veterans had to wait for a counselor in
order to inquire about the legitimacy of the survey. This posed two,
somewhat isolated, problems. The first is that veterans were only able
to get help with their questions during business hours, when customer
service representatives/benefits counselors were available. The second
is that a few VA customer service representatives were unaware of the
study, and hence unable to reassure respondents of its legitimacy.

####  *Developing a Brief and Effective Introduction to the Survey*

In our experience, most telephone survey refusals occur in the first few
seconds of the call. With that in mind, we scripted an introduction that
briefly stated who Westat is, the sponsoring agency, and the importance
of the study. With such an introduction, we intended to quickly provide
respondents with enough information to decide, at the very least, to
listen to and answer our screening questions (or in the case of the List
Sample, the verification question).

####  *Preparing a List of Frequently Asked Questions*

We developed a list of what we anticipated would be Frequently Asked
Questions about the NSV 2001, Westat's role in the study, and VA goals
(see Appendix D). We then required interviewers to learn the answers to
these questions well enough that they could talk about the study with
respondents naturally, instead of sounding like they were reading the
answers.

####  *Providing a Privacy Act Letter*

Some veterans requested written information about the study, saying they
would not participate until they received it. Within one business day of
such requests, we mailed a letter that described the NSV 2001 and
respondents' rights under the Privacy Act (see Exhibit 5-8) or a
background letter (Exhibit 5-4). We assigned a "mailout needed" code to
these cases and held them for two weeks before calling them again. We
mailed letters to 3,923 households over the course of data collection.

####  *Developing Standard Refusal Conversion Procedures*

If, at the initial contact, a respondent (or anyone in the household)
refused to participate in the survey, the CATI scheduler moved that
person's case to the refusal work class and put it on hold for a period
of time. After this "cooling off" period, the CATI scheduler assigned
the case to a refusal conversion interviewer. The cases of respondents
who refused again, after one conversion attempt, were held for another
cooling off period and released again. We rereleased some cases up to
two times. We did not attempt to recontact hostile refusers. Overall,
our rate of refusal conversion was 33.0 percent for screener cases, 80.9
percent for RDD Sample extended interviews, and 82.1 percent for List
Sample interviews.

####  *Mailing Letters to Refusals and Hard-to-Locate Veterans*

About halfway through data collection, we began mailing letters to cases
that had refused twice and cases that were hard to locate. The refusal
conversion mailout was conducted weekly from late July through late
October, 2001. We mailed a letter in a business size envelope with first
class postage, addressed "To the Family At" to RDD Sample cases with
whom we had been unable to make contact, or that had refused during the
household screening interview (see Exhibit 5-9). We sent a letter by
overnight delivery to List Sample and RDD Sample cases that we had been
unable to recontact for an extended interview, or that had refused to
complete an extended interview (see Exhibit 5-10 and Exhibit 5-11). Our
address matching vendor provided addresses for 70 percent of the RDD
Sample. We did not have reliable names for these cases, though, so we
addressed the letters "To the Veteran At." We personalized the List
Sample letters. Along with the letters to extended interview refusers
(or hard-to-locate cases), we enclosed the same letter from the
Secretary of the VA that we included in the advance mailing to List
Sample members. Between 2 and 5 days after we mailed the letters, we
made available for calling these

Exhibit 5-8. Privacy Act statement

> September 5, 2001

«TRC_ID»

«F_NAME» «L_NAME»«SUFFIX»

«ADDR1A»

«ADDR1B»

«CITY1», «STATE1» «ZIP1»

Dear «F_NAME» «L_NAME»«SUFFIX»:

Recently you were contacted by a Westat interviewer as part of a study
being conducted for the Department of Veterans Affairs. At that time you
stated you would like to receive a copy of the Privacy Act Statement
that applies to this study. The text of this statement is provided
below.

**Privacy Act Statement** -- ***National Survey of Veterans 2001***

[Authority]{.underline}: 10 U.S.C 2358: E.O.9397

**[PRINCIPAL PURPOSE]{.underline}:** This survey collects [veteran
current]{.underline} demographic and socioeconomic characteristics,
veteran health status, medical services usage and needs, health
insurance coverage, and veteran knowledge about and use of various VA
programs. Social Security Numbers are requested and will be used for two
purposes. First, it will be used to ensure that each respondent is
interviewed only once. Second, it will be used to obtain additional
information from VA files and other sources to better understand veteran
needs.

**[ROUTINE USES:]{.underline}** None.

**[DISCLOSURE:]{.underline}** Voluntary. Maximum participation is
encouraged so that data will be complete and representative, but there
are no adverse consequences for individuals who decline to participate.

I hope that the information provided here answers any questions you have
regarding this study. Should you wish to obtain additional information
you may contact:

Department of Veterans Affairs

Office of Policy and Planning

Attn: Ms. Susan Krumhaus 008A Phone: (202) 273-5108

810 Vermont Avenue, NW

Washington, D.C. 20420

If you need general information on programs and benefits offered by the
Department of Veterans Affairs, you may call their toll-free hotline:
**(800) 827-1000**.

Thank you for your assistance in this important study.

> Sincerely,

![](media/image4.png){width="1.698611111111111in"
height="0.6583333333333333in"}

> John C. Helmick, Jr.
>
> Senior Study Director

Exhibit 5-9. Screener refusal letter (RDD Sample)

WESTAT

National Survey of Veterans

1650 Research Blvd. Rockville, MD 20850-3129 301 251-1500 FAX 301
294-2040

> «DATESENT»

ID #«MAILID»

To the Family At

«ADDRESS»

«CITY», «STATE» «ZIP»

Dear Respondent:

**We need your help.** Recently, one of our telephone interviewers
called and asked you or someone in your household to take part in a
survey about veterans of the United States military. At this time, we
have not finished an interview with you. This survey is sponsored by The
Department of Veterans Affairs (VA). It will help the VA to evaluate
their current policies, programs and services.

Your answers to this survey will help us determine whether there are any
veterans in the household. We can't talk to every person in the country.
Instead, we scientifically selected a few people to represent the nation
as a whole. This means no other person can take your place. Your
individual participation will have a significant impact on the survey's
outcome. Thus, even if there are no veterans in your household, we would
still like to speak with you briefly.

I want to assure you that the information you provide is protected under
the Privacy Act and section 5701 of Title 38 of the U.S. Code.

We would be happy to speak with you at a time that is convenient for
you. If you would like to begin an interview or schedule an appointment,
please call Westat toll-free at 1-888-258-2194. Ask to speak with the
Veterans Survey Manager.

Thank you. We greatly appreciate your help.

> Sincerely,
>
> ![](media/image2.png)
>
> John C. Helmick
>
> Project Director

Exhibit 5-10. Extended refusal letter (List Sample)

WESTAT

National Survey of Veterans

1650 Research Blvd. Rockville, MD 20850-3129 301 251-1500 FAX 301
294-2040

> «DATESENT»

ID #«MAILID»

«NAME»

«ADDRESS»

«CITY», «STATE» «ZIP»

Dear «LOWERNME»:

**We need your help.** Recently, one of our telephone interviewers
called and asked you to take part in the National Survey of Veterans. At
this time, we have not finished an interview with you. This survey is
sponsored by the Department of Veterans Affairs (VA). It will provide
valuable information for planning benefits and services for all
veterans.

While your participation is voluntary, you are one of only a few
thousand randomly selected veterans whose answers will represent all 25
million veterans. Your individual participation will have a significant
impact on the survey's outcome. We cannot interview another veteran in
your place. Your answers, and those of other participating veterans,
will give the VA an up-to-date picture of the whole U.S. veteran
population.

I want to assure you that the information you provide is protected under
the Privacy Act and section 5701 of Title 38 of the U.S. Code. The VA
will use the information you provide to evaluate current VA policies,
programs and services for veterans and in deciding how to help veterans
in the future.

We would be happy to arrange an interview at a time that is convenient
for you. If you would like to begin an interview or schedule an
appointment, please call Westat toll-free at 1-888-258-2194. Ask to
speak with the Veterans Survey Manager.

Thank you. We greatly appreciate your help.

> Sincerely,
>
> ![](media/image2.png)
>
> John C. Helmick
>
> Project Director

Exhibit 5-11. Extended refusal letter (RDD Sample)

WESTAT

National Survey of Veterans

1650 Research Blvd. Rockville, MD 20850-3129 301 251-1500 FAX 301
294-2040

«DATESENT»

ID #«MAILID»

To the Veteran At

«ADDRESS»

«CITY», «STATE» «ZIP»

Dear Veteran:

**We need your help.** Recently, one of our telephone interviewers
called and asked you to take part in the National Survey of Veterans. At
this time, we have not finished an interview with you. This survey is
sponsored by the Department of Veterans Affairs (VA). It will provide
valuable information for planning benefits and services for all
veterans.

While your participation is voluntary, you are one of only a few
thousand randomly selected veterans whose answers will represent all 25
million veterans. Your individual participation will have a significant
impact on the survey's outcome. We cannot interview another veteran in
your place. Your answers, and those of other participating veterans,
will give the VA an up-to-date picture of the whole U.S. veteran
population.

I want to assure you that the information you provide is protected under
the Privacy Act and section 5701 of Title 38 of the U.S. Code. The VA
will use the information you provide to evaluate current VA policies,
programs and services for veterans and in deciding how to help veterans
in the future.

We would be happy to arrange an interview at a time that is convenient
for you. If you would like to begin an interview or schedule an
appointment, please call Westat toll-free at 1-888-258-2194. Ask to
speak with the Veterans Survey Manager.

Thank you. We greatly appreciate your help.

> Sincerely,
>
> ![](media/image2.png)
>
> John C. Helmick
>
> Project Director

screener and extended cases. RDD Sample cases for which we were unable
to obtain an address received the telephone followup without a mailout.
The release of the refusal conversion mailout cases occasionally
overlapped with the release of new sample. This created a huge volume of
work such that the refusal cases sometimes did not get called until up
to several weeks after they received the letters. Table 5-3 shows the
number of RDD screener, RDD extended, and List Sample extended interview
cases to which we mailed letters, and the percentage of those cases for
which we were able to complete an interview.

Table 5-3. Results of mailout to refusal and hard-to-locate cases

  --------------------------- --------------------- ----------------------
  Interview Type              Number of cases to\   Percent of mailed
                              which a letter was    cases\
                              mailed                with a completed
                                                    interview

  Screener interview          28,482                2.2

  RDD Sample extended         2,746                 20.6
  interview                                         

  List Sample extended        1,571                 28.4
  interview                                         
  --------------------------- --------------------- ----------------------

####  *Matching Deceased Veterans to the SSA Date-of-Death File*

During the course of data collection, we noticed that a large number of
List Sample veterans were reported as deceased either on the advance
letter update form or when we attempted to contact them over the
telephone. To ensure that we had the most accurate information possible
about our List Sample cases, toward the end of data collection we
matched all List Sample cases against the Social Security Administration
(SSA) Date-of-Death File. Through this effort, we were able to identify
317 cases of previously unknown status as deceased. We also confirmed
that 202 cases we had already identified as deceased were, indeed,
deceased.

####  *Retrieving RDD Screener Data*

For 24 RDD Sample cases, we discovered that some screened interview
information required to determine whether someone in the household was
eligible for an interview was missing. We called these households again
to collect the missing information. We retrieved the missing data in all
24 cases, identifying 13 of them as eligible for the study. As a result
of this effort, we completed seven additional interviews. The other six
eligible veterans refused to participate.

####  *Scheduling Additional Releases of Cases*

The CATI scheduler followed certain rules for determining when to close
cases with which we did not make contact. At the same time, we wanted to
ensure that we made every attempt to reach respondents who might have
been out of town or otherwise unavailable during our initial contact
attempts. Therefore, we programmed the CATI scheduler to automatically
release such noncontact cases after a designated "resting" period. Doing
so improved our chances of making contact with these difficult-to-reach
cases.

####  *Conducting End-of-Study Measures*

Toward the end of data collection, we used three additional methods to
increase the number of household contacts. First, on one day during each
of the last 4 weeks of data collection, interviewers left a message
every time they reached an answering machine. Second, 2 weeks before the
end of data collection, we increased the number of times we called
outstanding List Sample cases, of which there were 498. To do this, we
removed all outstanding (noncompleted) cases from the CATI scheduler. We
continued to try to make contact using a paper call record. Doing so
permitted us to identify the cases with which we had had little or no
contact, attempt to call them at times different from the previous
calls, and call multiple times during one day. Third, on the final
Saturday of data collection, we asked interviewers to let telephones
ring longer than usual before hanging up.