When selecting an intercultural assessment, it is critically important to investigate the reliability and validity of instruments you consider. Some assessments have been created by individuals who lack rigorous psychometric training and some do not provide peer-reviewed published evidence of their reliability and predictive validity. To be reliable, responses to scale items must be correlated to provide evidence that each dimension is measuring one thing at a time. Dimensions must be related but not so highly related that they do not assess different things. Additionally, the dimensions and the pattern of relationships among the dimensions should be stable across samples and across cultures.
Further, to be valid, a scale must have demonstrated, predictive validity. This means it must predict meaningful outcomes. A scale that aims to predict intercultural performance needs to include both self and observer assessments and should predict things like negotiating effectively across cultures, making quality decisions, adjusting to different cultural norms for time, authority, etc. Scales that were developed using only self-report data lack construct validity and should be avoided.
To summarize, examine published academic literature on the assessment being considered, make sure the instrument measures what they want to measure, and confirm that the tool is both reliable and valid. One very useful way to do this is to read published articles that compare different assessments. An example is the external review of ten cultural competence assessments conducted by intercultural psychologists David Matsumoto and Hyisung C. Hwang, published in the Journal of Cross-Cultural Psychology. In reviewing ten cultural competence assessments, the reviewers concluded that most of the tools lack validity and have unstable factor structures. However, the reviewers conclude there is “considerable evidence for the concurrent and predictive ecological validity” of CQ with samples from multiple cultures.
