Psychometric evaluation of the adaptation of the Verbal Cognitive Reflection Test in Bosnia and Herzegovina

Abstract

Cognitive reflection refers to the ability to suppress an incorrect initial intuition. This construct has traditionally been measured using numerical tasks, which led to the development of the verbal CRT-V test, aimed at more clearly distinguishing it from the domain of mathematical ability. In this study, the CRT-V test was examined on a sample of 318 participants from Bosnia and Herzegovina (45% female; 65% high school students, 35% university students). Along with the CRT-V, tests of verbal and matrix reasoning were administered. The one-factor CRT-V model exhibited excellent fit indices (CFI=.994; TLI=.992; SRMR=.060; RMSEA=.025, 90% CI [.006, .046]) and adequate convergent validity. However, several issues emerged. Specifically, item #9 had low factor loading and poor discrimination. Most items were too easy (70%+ success on 6/10 tasks), and test informativeness was mainly located below the average latent cognitive reflection trait level. Item #9 displayed bias (DIF) favoring males and high school students, while item #3 favored high school students and item #10 favored university students. Some items also contain terminological and cultural nuances not optimal for the multicultural BiH context. The one-factor CRT-V model showed good fit and satisfactory convergent validity. Average scores were moderately higher among university students compared to high school students, and slightly higher among women compared to men. Several psychometric issues were identified. Although the current version of the CRT-V may be useful, a more thorough revision or replacement of certain items is recommended to reduce bias and increase difficulty and informativeness, thereby enhancing the practical utility of the test.