Researchers find translation differences in French and English achievement tests
Written By: Ryan Smith
2004-12-23
Education researchers have discovered that 32 to 55 per cent of the questions on national achievement tests in Canada show discrepancies between the English and French versions."It's disconcerting how often we're finding these [discrepancies]," said Dr. Mark Gierl, a University of Alberta education professor and co-author of a paper on the subject recently published in Applied Measurement in Education.
Gierl believes that the discrepancies may occur because after the tests are translated —usually from English to French – no one evaluates the translated items to ensure that they measure mental capacity to the same degree as the original item.
"This is an area where Canada, a bilingual nation, should be leading the way, but right now we're not nearly as good as we should be," said Gierl, who was recently named the Canada Research Chair in Educational Measurement and was also recently hired to help develop the new SAT, the college application aptitude test in the United States.
For his research on the incompatibility of English and French tests in Canada, Gierl and his colleagues took the results of recent tests and then analysed the items – the questions and their multiple choice answers – that yielded significant differences in results between English and French students.
In some cases the difference between the French and English item was easy to spot, such as a simple translation or formatting error; in others, it was more complex. And sometimes the researchers concluded that the discrepancy in results was not caused by differences in the items but probably stemmed from genuine differences in the two groups being compared – likely the result of cultural or curriculum differences.
Gierl explained that item differences can be placed into one of four categories: omissions or additions that affect meaning; differences in words or expressions inherent to language or culture; differences in words or expressions not inherent to language or culture; and format differences.
An example of a format difference occurred in a recent math test for Grade 6 students, who were asked to calculate the difference between two times given in the text of a question. On the French exam, the times were given according to a 24-hour clock (5:20 and 20:15); on the English version, they were given in a.m. and p.m. (5:20 a.m. and 8:15 p.m.).
The added complexity of recognizing the difference between a.m. and p.m., combined with the fact that one of the incorrect answers given in the multiple choice test was close to the difference between the two times if they had both been in the a.m. or both been in the p.m., likely explains why a significant number of English students did not do as well on this question as their French counterparts did, Gierl said.
"Our goal as researchers in this field is to make achievement tests in different languages as fair and as comparable as possible," Gierl added. "We want to find out why [discrepancies] occur and then figure out ways to eliminate them and avoid them in the future."
Original: ExpressNews