State-Trace Analysis Meets Psychometrics

Why the Big Five Questionnaires are not Based on Five Latent Factors and How to Fix Them

Johannes Titz

Chemnitz University of Technology, Germany

2024-07-24

Why are available methods insufficient?

Factor analysis based on (Pearson) correlations is not a good way to analyze dimensionality because Pearson correlations do not reflect unidimensionality well.

State-trace analysis is the new kid on the block

State-trace analysis (Dunn et al., 2019; Dunn & Kalish, 2018; Dunn & Kirsner, 1988) is much better suited to test (uni)dimensionality hypotheses because:

  • it has only one premise: monotonicity between latent variables and manifest variables (very reasonable for psychology!)
  • it is easy to use: just plot two variables against each other and test for monotonicity
  • it is rigorous: mathematically proven

Example Liberalism and Intellect

In the Big Five model Liberalism and Intellect are facets of Openness for Experience. So they must be isotonically related.

Data: IPIP-NEO-120 with N=618,000 participants (Johnson, 2014; Kajonius & Johnson, 2019)

  • p<0.000018

  • p = .0127

  • p < 0.001

Just the tip of the iceberg

Overall, there are 267 violations of monotonicity in the IPIP-NEO-120. Furthermore, the NEO-PI-R suffers from similar problems.

Conclusion

State-trace analysis applied to the Big Five model reveals some problems that have not been identified yet with traditional methods. This is a good basis to improve psychological measurement beyond factor-analytic strategies.

Preprint

https://doi.org/10.23668/psycharchives.13972

Presentation

https://johannestitz.com/ICP2024

References

https://johannestitz.com/ICP2024

Dunn, J. C., Heathcote, A., & Kalish, M. (2019). Special issue on state-trace analysis. Journal of Mathematical Psychology, 90, 1–2. https://doi.org/gn2mjz
Dunn, J. C., & Kalish, M. L. (2018). State-trace analysis. Springer International Publishing. https://doi.org/10.1007/978-3-319-73129-2
Dunn, J. C., & Kirsner, K. (1988). Discovering functionally independent mental processes: The principle of reversed association. Psychological Review, 95, 91–101. https://doi.org/10.1037/0033-295X.95.1.91
Johnson, J. A. (2014). Measuring thirty facets of the five factor model with a 120-item public domain inventory: Development of the IPIP-NEO-120. Journal of Research in Personality, 51, 78–89. https://doi.org/bc99
Kajonius, P. J., & Johnson, J. A. (2019). Assessing the structure of the five factor model of personality (IPIP-NEO-120) in the public domain. Europe’s Journal of Psychology, 15(2), 260–275. https://doi.org/gh8gk3