When Translation Affects Research Outcomes

A recent article in the Journal of Consumer Research reveals an important finding – that survey results can be biased if the translations are word-for-word instead of meaning-for-meaning. “If the response category labels used in different languages are not equivalent, this could bias survey results,” explain the survey authors, who hail from Ghent University, Vlerick Business School, and Pennsylvania State University.

For example, the researchers found that in one consumer survey of French speakers, the response categories were more likely to be chosen if the translations were more common expressions – such as tout à fait d’accord (completely agree) versus extrêmement d’accord (extremely agree). Likewise, a survey carried out in Dutch showed that response rates increased when the translation used the Dutch equivalent of terms such as “completely agree” instead of less common terms like “strongly agree.”

This finding won’t come as a surprise to most language professionals. I recall as an interpreter that one of the hardest things to interpret was when doctors instructed patients, “Rate your pain on a scale of 1 to 10.” While this is not difficult to render from one language into another, culturally, it does not always make sense. Many patients had no idea how to rate their pain using this method. They had never been asked to do such a thing in their home countries, and the idea of giving a numerical score to pain seemed not only foreign, but completely strange.

One thing that designers of surveys destined for multilingual audiences can do is adapt their response choices instead of merely translating them. If they use similar scales (a five-point scale, for example) and look at the same factor (agreement or satisfaction, for example), the data can still be compared across language groups.

In the future, researchers may find themselves including a discussion of cultural and linguistic adaptation for each language or country in their methodology write-up, to prove that they are using terminology that is equally recognizable and acceptable in each market and language they are surveying. For that, they’ll certainly need help from their translation partners. It might not be enough for them to simply throw their content over the wall and wait for the translation. Going forward, they might also need a description of why certain terms were selected in lieu of others, in order to prove that the translations – and the research findings that result from them – can be trusted.

About Nataly Kelly

Nataly brings nearly two decades of translation industry experience to Smartling, most recently as Chief Research Officer at industry research firm Common Sense Advisory. Previously, she held positions at AT&T Language Line and NetworkOmni (acquired by Language Line), where she oversaw product development. A veteran translator and certified court interpreter for Spanish, she has formally studied seven languages, and is currently learning Irish. A former Fulbright scholar in sociolinguistics, Nataly lives in the Boston area with her husband and daughter. When she isn’t working, you’ll usually find her translating Ecuadorian poetry, writing books, and exploring the world (36 countries and counting!).