State-Trace Analysis Meets Psychometrics
Why the Big Five Questionnaires are not Based on Five Latent Factors and How to Fix Them
Chemnitz University of Technology, Germany
2023-05-23
Why are available methods insufficient?
Factor analysis based on (Pearson) correlations is not a good way to analyze dimensionality because:
- untestable, unrealistic assumptions are made (e.g. linear relationship between latent variables and manifest variables)
- Pearson correlations do not reflect unidimensionality well
State-trace analysis is the new kid on the block
State-trace analysis (Dunn et al., 2019; Dunn & Kalish, 2018; Dunn & Kirsner, 1988) is much better suited to test (uni)dimensionality hypotheses because:
- it has only one premise: monotonicity between latent variables and manifest variables (very reasonable for psychology!)
- it is easy to use: just plot two variables against each other and test for monotonicity
- it is rigorous: mathematically proven
Example I: Liberalism and Intellect
In the Big Five model Liberalism and Intellect are facets of Openness for Experience. So they must be isotonically related.
Data: IPIP-NEO-120 with N=618,000 participants (Johnson, 2014; Kajonius & Johnson, 2019)
Example II: “I like to take it easy”
The Item “I like to take it easy” from the IPIP-NEO-120 is supposed to reflect little Extraversion (being inactive), but is not always interpreted this way. “Taking it easy” is sometimes associated with “being relaxed”, “easy-going” or even “cool”.
Same data set with N=618,000.
Just the tip of the iceberg
Overall, there are 267 violations of monotonicity in the IPIP-NEO-120. Furthermore, the NEO-PI-R suffers from similar problems.
Conclusion
State-trace analysis applied to the Big Five model reveals some problems that have not been identified yet with traditional methods. This is a good basis to improve psychological measurement beyond factor-analytic strategies.