Guideline Implementation: Translating a New 

Canadian/U.S. Health Survey into Spanish. 

Deborah Rose, Ph.D. 

National Center for Health Statistics, 3311 Toledo Road, Hyattsville, MD 20782, DROSE@cdc.gov 

 

Background 

Articles on the translation of health surveys and medical scales use three general approaches. The first approach emphasizes 

the qualitative procedures used to produce a translation (1,2,3). It involves multiple reviews that can be sequential or 

simultaneous. The second focuses on quantitative analysis, by making psychometric comparisons between the scores 

produced by the two language versions of a scale (4,5). The third, used less often, tests validity by comparing the scores 

obtained on a symptom scale with diagnostic evaluations or tests by a clinician (6,7,8). Most articles report using some 

combination of these three approaches. In addition, a few first discuss the relevant theoretical aspects in depth before 

covering the practical, methodological ones (9,10). 

Most of the articles reviewed covered the translation of a defined, English-language scale that was administered to a small, 

relatively homogeneous patient group. In contrast, the new methodology used in this effort applies to large, multi-topic 

health or demographic surveys that are administered to heterogeneous, nationally representative samples of the U.S. 

population on a repeating basis. Such surveys do not usually contain standardized, psychometrically validated scales of the 

kind described in the literature, so expanded methodology is needed to cover such surveys. 

In November of 2001, the Census Bureau convened a panel that included experts from federal agencies in the United States 

and Canada, private survey research firms, professional translators, and international consultants. They were charged with 

developing a set of guidelines and best practices to serve as a model for the translation of large-scale health and demographic

surveys from English into Spanish and other languages. The resulting guidelines took the methodological approach, rather 

than psychometric analysis, or clinical validation. They are outlined in Translation Guidelines and Best Practices for the 

Translation of English Language Demographic Surveys into Other Languages by Manuel de la Puente, elsewhere in this 

volume. 

The first use of these guidelines was to translate a new joint Canada/U.S. health survey from English into Spanish. Steps 

included: 1) selection of the contractor, 2) review of the translation, 3) adjudication, 4) development of the survey instrument 

in three languages, 5) a field pretest of the survey, 6) selection and training of interviewers; and, 7) incorporation of feedback 

from the field in revising both questionnaires. This paper* will review the successes and challenges of putting the guidelines

into practice for a bi-national health survey. 

Methods

The Joint Canada/United States Health Survey (JCUHS) was a collaboration between the governments of Canada (Statistics 

Canada) and the United States (National Center for Health Statistics) to carry out a single survey in both countries to 

facilitate comparisons of the health and health care of the two populations. It was a one-time telephone survey, developed 

between 2001 and 2002, and administered by Statistics Canada from their offices in Canada, between the fall of 2002 and 

summer of 2003. Households were selected by random digit dialing (RDD). The sample size was 8,700 (5,200 from the 

United States, and 3,500 from Canada), with one adult interviewed per household. 

__________________ 

*

 This paper accompanied a presentation given at the 2003 conference of the Federal Committee on Statistical Methodology (FCSM), Concurrent Session: 

Linguistic and Cultural Issues in Survey and Questionnaire Design, Tuesday, November 18, 2003, in Arlington, VA. 

Page 24
The new survey incorporated questions from the Canadian National Population Health Survey (NPHS), the Canadian 

Community Health Survey (CCHS), and the U.S. National Health Interview Survey (NHIS). As these were ongoing health 

surveys in each country that were already administered in two languages (English and French in Canada; English and Spanish 

in the United States), the decision was made to use all three languages in the joint survey (English, French and Spanish). 

Staff members from Statistics Canada handled production of the French version of the joint survey. Staff members from 

NCHS handled the translation process for the Spanish version of the joint survey. This presentation will focus on the Spanish 

translation process. 

A few definitions are helpful before describing the procedures. The “source language” is the language of the original text, in

this example, English. The “target language” is the language into which the text is translated, in this example, Spanish. 

“Interpreting” is informally rephrasing a source language speech or text into speech in the target language. Other terms for 

interpreting include “sight”, “simultaneous” or “on-the fly” translations. The result is a spoken, impermanent version of the 

translated text. “Translating” is the formal process of converting a written text from the source language into a written text in 

the target language, using glossaries, examples, expertise, experience, dictionaries, review, and testing. The product is a 

permanent, written version of the text, in the target language. 

 

 

Pre-translation Preparation

Recommended. Before beginning a new translation, the expert panel suggested: 1) determining the style and language level 

needed for the survey, 2) collecting examples of similar surveys, and 3) developing a glossary of specialized terms. 

Actual. The JCUHS followed the first two recommendations but not the third. The staff in the agencies of both countries 

had long experience in developing surveys using clear, correct language, in a structured conversational style, targeted to the 

7

th or 8th grade literacy level, so that people with low levels of education could understand and respond to the questions. 

These already fielded questionnaires formed the basis of the questionnaire for the new survey. 

The two Canadian questionnaires (NPHS and CCHS) were available in English and French, and the 2000 U.S. NHIS was 

available in English and Spanish. The translators did not assemble a separate glossary of health, governmental and medical 

terms. But, they did use translated terms from the existing U.S. questionnaires, which made the standardization implicit 

rather than explicit. The Canadian questionnaires were not translated into Spanish, so items selected from the Canadian 

surveys did not have an existing Spanish translation. 

 

Selection of the Contractor

Recommended. The expert panel suggested employing professional firms, with professional translators who had subject 

matter expertise, rather than using bilingual staff members without professional translation credentials. It recommended 

reviewing the credentials of the actual staff members who would be doing the translation. 

Actual. The JCUHS was able to follow these recommendations, and found a small translating firm in which the principal 

was a native speaker of Spanish, with a university education in Spanish, graduate work in English, and extensive experience 

translating health and demographic surveys. 

Translation Process

Recommended. The expert panel recommended beginning with a finalized document in the source language to minimize the 

number of translation versions needed (3). 

Actual. Although three existing English language questionnaires were used to compose the survey, it was still necessary to 

select subsets of questions and synchronize the whole to create a coherent new questionnaire in English. Unfortunately, due 

to time constraints, for many topics, the survey instrument was not finalized in English before the review. Questionnaire 

development in English and translation into Spanish had to occur at the same time, which increased the difficulty of 

providing a final Spanish version in advance of the translation review conference. 

Page 25
Review of the Translation

Recommended. The expert panel suggested several alternate methods to translate and review the translation such as: 

translation by committee; having two different people translate the whole questionnaire; having two or more people translate 

alternate sections of the questionnaire. 

For one qualitative method, the recommendations of the expert panel differed sharply from the recommendations in the 

literature. Nearly all of the translations reported in the literature relied heavily on a “forward/backward” technique in which

one or more translators translated the English text into Spanish, and a different set of translators translated the Spanish 

version back into English. The expert panel strongly recommended that back translation not be used for languages in wide 

use, because it is a crude kind of check compared with committee review and field testing. For rare languages with few 

informants, back translations might be used of necessity, but should be regarded with great caution, as mis-translations could 

occur at either translation step, or could even cancel each other out, without such errors being detected by the survey authors. 

Actual. The JCUHS used the simultaneous committee approach, reviewing the questionnaire in English and Spanish on a 

question-by-question basis. The expert committee was composed of the translators, experienced bilingual U.S. Census 

interviewers, and bilingual survey and subject matter experts from two federal agencies. By origin and experience, the bilingual reviewers represented several Hispanic subgroups including: Mexican, Mexican-American, Puerto Rican, Central 

American and South American. A bilingual French-English observer from Statistics Canada also attended the review. The 

French version of the questionnaire was consulted occasionally to aid in the review of the Spanish. 

This was the first time that the translators attended the NCHS/Census review committee meetings. The U.S. questionnaire 

designers and reviewers found it very valuable to be able to discuss the translation directly with the translators. 

Adjudication between Alternatives

Recommended. The committee recommended adjudication by a project manager with the authority to make the final 

decision as a separate step for situations where the reviewers were not able to come to consensus on particular items. 

Actual. For the JCUHS, a separate adjudication stage did occur, but not because of lack of consensus among committee 

members. Rather, because of the tight time frame, the face-to-face committee review occurred early in the translation cycle, 

while the translation of newly drafted sections of the questionnaire continued for several months. Translation and 

transmission of both the English and Spanish sections continued by electronic mail after the end of the committee review 

sessions. The translation project managers of both countries coordinated closely in collecting, allocating and tracking the 

many iterations of the questionnaire pieces in English and in Spanish. 

The U.S. based translation review committee did note a stylistic difference between the JCUHS translation and the field 

version of the NHIS. Over the years that the members of this committee have worked together, they have developed a 

concise, yet conversational style that communicates the essential concepts and wording of the original. It uses a syntax that is 

correct, but more informal, to assure that the questionnaire can be understood and answered by respondents with a low level 

of education. 

Because the translators of the JCUHS were educated in Spanish at the university level, the Spanish translation of the joint 

survey tended to be more formal than that used in the NHIS. Had there been more time, the ideal would have been to test the 

translation in the field, and integrate the two approaches. 

Developing the Survey Instrument in All Interview Languages

Recommended. The expert panel recommended that the same interview format be used for all languages of the interview, 

and that, to the extent possible, they be programmed simultaneously. The panel recommended the design of a separate screen 

for each question in each language, to readily allow for linguistic differences in grammar, syntax and word order. The panel 

also recommended that the interviewer be able to switch between languages on a question-by -question (or screen-by-screen) 

basis, as needed. Interviewers can use this capability during the interview to verify the phrasing or meaning of technical 

terms in the original language, when necessary. 

Page 26
Actual. Statistics Canada already had a wealth of experience in developing a dual language (English and French) Computer 

Assisted Telephone Interview system (CATI) using Blaise survey software (Windows version). It extended this technology 

to add the third language, Spanish. Pressing the F2 function key allowed the interviewer to cycle between English and 

French or English and Spanish for each screen. 

Field Pretest of the Survey

Recommended. The expert panel recommended a full field pretest of the complete survey, in all of the languages to be 

used. A complete field test includes: developing and using training materials, conducting interviews with households from 

the general population, debriefing the interviewers, debriefing the observers, and revising the questionnaires to take into 

account the improvements suggested. 

Actual. There was not enough time to field a complete field test of the joint survey. Observers from Statistics Canada and 

NCHS attended the interviewer training and/or the first week of interviews, but the translators and reviewers were not able to 

attend. There were no formal debriefings, and there was no opportunity to revise the questionnaires between the test and the 

beginning of full field operation. 

Selection of Interviewers

Recommended. After some discussion, the expert panel recommended that potential bilingual interviewers be tested in

reading, writing and speaking both languages. Even for Spanish, which has a simplified, phonetic, written form, those with 

no formal education in Spanish are not able to read some sounds properly. 

Actual. The entire joint survey was carried out by telephone from Canada, using Canadian interviewers. Two regional 

offices carried out the Canadian interviews (Manitoba and Quebec). Two other regional offices carried out the United States 

interviews (British Columbia and Ontario). Canadian respondents were interviewed in English or French, and United States 

respondents were interviewed in English or Spanish. Those who interviewed the Canadian sample were fluent in English, 

French, or both. Those who interviewed the United States sample were fluent in English, and some were also fluent in 

Spanish (11). 

Training of Interviewers

Recommended. The expert panel recommended that at least a portion of the interviewer training be completed in both 

languages. For the bilingual interviewers, training in the second language should include a discussion of any special aspects 

of the second language that might affect the interview, and several complete practice interviews in both languages. 

 

Actual. For the Canadian portion of the survey, the training was in English in Manitoba, and in French in Quebec. For the 

United States portion of the survey, training was entirely in English. Some general tips were given on the health care system 

in the United States, but there was no formal training on the Spanish version of the questionnaire, and structured practice 

interviews in Spanish were not included. 

Feedback from the Field

Recommended. In addition to feedback obtained during a field pretest, some of the articles on translation suggested that the 

interviewers be surveyed during regular operations, to see if they had any consistent problems in administering the translated 

questionnaire. 

Actual. In June 2003, Statistics Canada sent a debriefing questionnaire to its field interviewers that included the following 

questions: 

o For those interviewers who conducted the interview in Spanish, did you come across any complications, i.e. 

different dialects that may not have been understood? 

o Were language barriers an issue (for example, being understood and understanding the respondent’s unfamiliar 

accents)? 

The responses to these questions are not yet available. 

Page 27
Other translation needs that emerged from this experience included: 

o The desirability for early cognitive testing in both languages 

o The need for translations of additional materials such as: letters, household rosters, entry and exit statements, 

 hidden or default responses, and help screens 

o Allowing enough time for the translation and review process to improve the text in both languages 

o Need for paper questionnaires in both languages, as archival resource, and as models for future surveys 

o Documentation of the translation process 

Difficulties of the collaboration included: problems in priorities, timing and implementation caused by the split 

responsibilities for design (both countries) and field implementation (Canada only), the tight time frame, the decreased 

budget, and the need to adhere to the regulations of two different countries. Unlike the U.S. NHIS translation review, for the 

Joint Survey, the Spanish translators and reviewers had no contact with the Canadian interviewers, so could not take 

advantage of their expertise. Benefits of the collaboration included: the experience Statistics Canada already had in 

producing computerized surveys in multiple languages, and the collegiality between the survey developers of both countries. 

Recommendations

Should the Guidelines be changed? No. The recommendations of the panel members reflected years of experience. The 

panel members learned from each other, and included all the steps that each felt were vitally important. Should the 

implementation be changed? Yes. Future translations should allow more time, more money, and should follow the 

Guidelines more closely. 

What should survey designers focus on if full implementation of the Guidelines is not possible? While following all of the 

recommended steps will produce the most appropriate translation of a survey for the target population, if that is not possible,

each of the steps selected should be carried out as fully as possible, with attention paid to compensating for those left out. 

For example, in this case study, following the recommendations for selecting a qualified translator and an experienced review

committee probably avoided the most serious translation problems, even in the absence of a full field pretest. If those two 

recommended steps could not be followed, then a full field pretest, interviewer and observer debriefing, and reformulation of 

the questionnaires would be essential to counterbalance the lack of a qualified translator and experienced review committee. 

Conclusions

This paper outlines the first effort to use the guidelines developed by an expert panel convened by the US Census Bureau in 

2001, to translate, test and implement the Spanish version of a new English language questionnaire. The suggestions for 

carrying out and reviewing the Spanish translation were implemented to the extent possible within a very short time frame. 

The suggestions on training were implemented for the French version, because the Canadian Federal Government has two 

official languages and requires the same procedures for both. Because of severe time and budget restraints, it was not 

possible to fully implement the guidelines for testing or training for the Spanish version of the questionnaire. According to

the literature, this pattern is not unusual, either due to the same constraints, or because a completely dual system is not 

considered from the start. 

Page 28
References

1. Weidmer B, Brown J, Garcia L. Translating the CAHPS 1.0 Survey Instruments into Spanish. Consumer 

Assessment of Health Plans Study. Med Care. 1999 Mar; 37(3 Suppl): MS89-96. 

2. Cella D, Hernandez L, Bonomi AE, Corona M, Vaquero M, Shiomoto G, Baez L. Spanish language translation and 

initial validation of the functional assessment of cancer therapy quality-of-life instrument. Med Care. 1998 Sep; 

36(9): 1407-18. 

3. Institute of Translation and Interpreting. Translation -- getting it right. A guide to buying translations. Milton 

Keynes, England. 2002. (Web address: www.iti.org..uk). 

4. Arias, I, Rodriguez E, Padilla J, Gonzalez N, Rodriguez MA. Translation to Spanish, reproducibility, and crosscultural adaptation of the Miler-Rahe Recent Life Change Questionnaire in Venezuela. Arthritis Care Res. 1999 

Aug 12(4):287-93. 

5. Azocar F, Arean P, Miranda J, Munoz RF. Differential item functioning in a Spanish translation of the Beck 

Depression Inventory. J Clin Psychol. 2001 Mar; 57(3): 355-65. 

6. Arocho R, Kason NM, Colon B, McMillan CA. Translation and validation of the American Urological Association 

symptom index into Spanish. Clin Ther. 1995. July- Aug. 17(4):777-85. 

7. Leavitt F, Gilbert NS, Mooney V. Development of the Hispanic Low Back Pain Symptom Check List. Spine. 1994 

May 1; 19(9): 1048-52, discussion 1052-3. 

8. Perpina M, Belloch A, Marks GB, Martinez-Moragon E, Pascual LM, Compte L. Assessment of the reliability, 

validity and responsiveness of a Spanish Asthma Quality of Life questionnaire. J Asthma. 1998; 35(6):513-21. 

9. Reynoso GA, March AD, Berra CM et al. Development of the Spanish Version of the Systematized Nomenclature 

of Medicine: Methodology and Main Issues. Proc AMIA Symp. 2000: 694-8. 

10. Pasick, RJ, Stewart SL, Bird JA, D’Onofrio CN. Quality of data in multiethnic health surveys. Public Health Rep. 

2001: 116 Suppl 1:223-43. 

11. Statistics Canada. Joint Canada - United States Health Survey (JCUHS). Interviewer’s Manual, October 2002. 

Ontario, Canada. 

Page 29