Backtranslation vs. Committee Approach: An 

Experiment Comparing How They Perform in 

Questionnaire Translation 

FCSM Conference 

Washington DC 

November 5, 2013 

Alisú Schoua-Glusberg Ana Villar 

Research Support Services City University, London 

alisu@researchsupportservices.com Ana.Villar.1@city.ac.uk 

Research Support Services - 2013
1 

Outline 

 Background 

 Research goals 

 Research design 

 Findings 

 Lessons learned and next steps 

Research Support Services - 2013
Research Support Services - 2013 2 

BACKGROUND 
Survey Translation 

1. Increasing need for multilingual 

surveys 

2. Translation usually outsourced 

3. Need to make sure translation is 

fit for use 

4. Need to take appropriate 

assessment steps 

Research Support Services - 2013 3 
Research Support Services - 2013 4 

What is survey translation 

assessment meant to do? 

1. Judge adequacy of text for culture 

2. Judge appropriateness of translation 

 Misunderstandings/translation mistakes 

 Regional variations, tone adequacy 

 Check for grammar errors/typos 

3. Provide alternatives where mistakes 

are encountered  find solution 

4. Evaluate how question is understood 
Research Support Services - 2013 5 

Translation Assessment Methods 

1. Expert review 

2. Committee review 

3. Focus groups 

4. Cognitive interviewing 

5. Comparison of original text to a 

back-translation 
Research Support Services - 2013

Translation Assessment Methods 

1. Expert review 

2. Committee review 

3. Focus groups 

4. Cognitive interviewing 

5. Comparison of original text to a 

back-translation 

6 
Research Support Services - 2013 7 

Back translation (BT) 

1. A text translated into language B 

from language A is translated back 

into language A. 

2. The two language A texts are 

compared to try to decide on the 

quality of the language B text. 

3 The non-English B text is NOT 

assessed in standard BT 

assessment 
Research Support Services - 2013 8 

What is Committee Approach? 

(Also known as Team Approach) 

A team of translators works together: 

 They TRANSLATE the instrument 

independently (full or split) 

 They meet to REVIEW and refine the initial 

translation(s) 

 Present at the meeting one or more persons 

who can ADJUDICATE disagreements 

Thus, they implement first steps of Harkness’ 

TRAPD model. 
Research Support Services - 2013 9 

RESEARCH GOALS 
Research Support Services - 2013 10

Research goals 

 Gather evidence on efficiency of 

Back translation 

Committee approach 

 Does each approach catch as many 

translation problems/issues as the 

other? 
Research Support Services - 2013 11

RESEARCH DESIGN 
Research Support Services - 2013 12

Background Information 

 Procedures we chose mirror standard 

practice in BT and CA based on large 

literature review and our own practice 

 We used Folkman & Lazarus’ 66 item 

Ways of Coping Scale, widely used 

questionnaire containing a wide range of 

thoughts and acts that people use to 

deal with the internal and/or external 

demands of specific stressful 

encounters. 
Sample Items 

Ways of Coping Scale (Folkman & Lazarus) 

Please read each item below and indicate, by using the 

following rating scale, to what extent you used it in the 

situation you have just described. 

(Not used/Used somewhat/Used quite a bit/Used a great deal) 

1. Just concentrated on what I had to do next – the next step. 

2. I tried to analyze the problem in order to understand it 

better. 

3. Turned to work or substitute activity to take my mind off 

things. 

4. I felt that time would make a difference – the only thing to 

do was to wait. 

Research Support Services - 2013 13
Research Support Services - 2013 14

Design for Committee 

Approach 

 Followed full committee (vs. split) so 

there would be one full original 

translation to send to backtranslator 

 Each of 3 English-Polish translators 

worked independently 

 Committee discussions followed 

 Fully reviewed and adjudicated final 

version was produced 
Research Support Services - 2013 15

Design for BT 

 Of the 3 translators, we chose translator with 

strongest credentials 

 Translation from that translator was given to 

backtranslator 

 BT was reviewed by research team as would 

typically be done in BT, by comparing with 

original English 

 For discrepant items, comparison of original 

English and BT was done with another Polish 

translator. Errors or problems in BT were 

identified. Errors or problems in original 

translation were identified. 
Research Support Services - 2013 16

Compare Both Methods 

1. Identify problems in the original 

translation: 

 that were found by BT process 

 that were missed by BT process 

2. Identify problems in the original 

translation: 

 that were found by CA process 

 that were missed by CA process 
Research Support Services - 2013 17

FG with Monolinguals 

We held a focus group with Polish monolinguals (Polish) to get their views on most 

idiomatic version in cases where style differs in 

OT and CT 

12 participants (7 women, 5 men) / Ages 24 to 

69 / Time in US ranged from weeks to 19yrs 

Analysis still to be completed 
Steps Completed to Date 

1. Original Translation (OT) + Back translation 

(BT) 

2. Committee translation (CT) 

3. Identification of OT issues caught by BT 

4. Identification of issues created by BT where 

OT was not a problem 

5. Identification of OT issues caught by CT 

6. Identification of OT issues missed by BT 

7. Identification of issues created by CT 

8. Preliminary look at FG findings 

Research Support Services - 2013 18
Research Support Services - 2013 19

FINDINGS TO DATE 
Findings on BT 

 Comparison of Original English (OE) 

and Backtranslated English (BT) 

showed: 

1. Of the 66 items, 31 appeared fully 

consistent in meaning 

2. In the 35 items investigated: 

 12 showed problems in 1st step (OT) 

 12 showed problems in 2nd step (BT) 

 5 were not really problems 

 6 need further investigation because of use of 

idioms or because of ambiguity in original 

Research Support Services - 2013 20
Research Support Services - 2013 21

Example: BT error (false 

positive) and OT error 

 Stood my ground and fought for what 

I wanted (OE) 

 Postawiłem na swoim i walczyłem o 

to, co chciałem osiągnąć.(OT)

 [I stood my ground and fought for 

what I wanted to achieve.] 

 I stood up for myself and fought for 

what I wanted done. (BT) 
Research Support Services - 2013 22

Findings on BT 

 Comparison of Original English (OE) 

and Backtranslated English (BT) 

showed: 

 BT implies problems with target that 

are not problems 

 BT will find some possible problems 

 BT does not identify source of problem 

or its resolution 

 Further steps needed, including target 

language expertise 
Research Support Services - 2013 23

Preliminary findings on CT 

 CT found 10 instances where 

semantically BT did not identify 

problems but where the OT was found 

to be unidiomatic (stilted / translation 

sounding) 

 CT appears to have found all 

problems that BT identified 
Research Support Services - 2013 24

Initial FG Findings 

 Not unexpectedly, a lot in language is 

idiosyncratic 

 Looked at first 10 items. On 2, all OT 

and BT meant the same. On other 8, 39 

said yes and 55 said no. (What does this 

say about the final BT step? 

 Participants equally divided as to which 

sounds more idiomatic. 

 More reasons to pretest translations 
Research Support Services - 2013 25

Next steps 

Our findings point to limitations of BT: it 

creates as many problems as it finds 

 Finish analysis to compare methods 

 Include monolinguals in assessing most 

idiomatic version 
26

Recommendations 

 If committee translation involves 

an adjudicator who is not a speaker 

of the target language, add step in 

committee meetings where CT of 

each item is glossed back into the 

source language. 

Research Support Services - 2013
Research Support Services - 2013 27

Thank You 

 Questions 

 Email: 

alisu@researchsupportservices.com