Rating versus ranking: What is the best way to reduce response and language bias in cross-national research?

Abstract

We propose solutions to two recurring problems in cross-national research: response style differences and language bias. In order to do so, we conduct a methodological comparison of two different response formats-rating and ranking. For rating, we assess the effect of changing the commonly used 5-point Likert scales to 7-point Likert scales. For ranking, we evaluate the validity of presenting respondents with short scenarios for which they need to rank their top 3 solutions. Our results - based on two studies of 1965 undergraduate and 1714 MBA students in 16 different countries - confirm our hypotheses that both solutions reduce response and language bias, but show that ranking generally is a superior solution. These findings allow researchers to have greater confidence in the validity of cross-national differences if these response formats are used, instead of the more traditional 5-point Likert scales. In addition, our findings have several practical implications for multinational corporations, relating to issues such as selection interviews, performance appraisals, and cross-cultural training. (C) 2009 Elsevier Ltd. All rights reserved.