The pseudo-paradigmatic science

Johannes Titz and Matthias Hörr

Motivation

• The replication and confidence crisis in psychology has prompted numerous debates about improving research practices.
• However, much like in psychological research itself, these debates are atheoretical.
• To engage in a meaningful debate, a theory of scientific development is essential.
• Kuhn’s theory of scientific progress provides an alternative perspective on the challenges faced in psychology.
• Today: Focus on a diagnosis rather than recommendations for improvement.

15+40 It is not possible to give meaningful recommendations for improvement in a 30-min talk; the idea here is to give a perspective on the debates in psychology and its crises.

Interpretation of Kuhn’s philosophy of science

2:15

What most of us probably know is that kuhn differentiates between pre-paradigmatic and paradigmatic sciences.

Exemplars

• “Universally recognized scientific achievements that provide model problems and solutions to a community of practitioners” (p. xliii, Kuhn, 2012–1962).
• These serve as educational tools for students, encompassing both theoretical and empirical knowledge.
• But what is an exemplar, really?

25 This is very important in Kuhn’s theory because through this process of acuqiring the foundation of the discipline they become members of the community.

One exemplar in physics: pendulum experiment

• This experiment is a key part of the empirical experimental project in a physics curriculum (Felgner et al., 2021, p. 14).
• It is a fundamental experiment that every student is required to complete.
• The experiment involves calculating the gravitational acceleration on Earth (\(g\)) by measuring the length of a pendulum and its period of oscillation.

30

Pendulum experiment: characteristics

• Involves substantial theoretical development (“rich in theory”).
• Symbolic derivations, incorporating differential equations, establish a link to empirical observations, exemplified by the relationship \(T = 2\pi\sqrt{\frac{L}{g}}\) under certain conditions.
• Practically deterministic from a psychological perspective, as indicated by a high coefficient of determination \(R^2 = 0.9998\) (e.g., Milster, 2009).

1:30

Tangential Force Equation

The tangential component of the gravitational force provides the restoring force: \(-mg \sin \theta = mL \frac{d^2\theta}{dt^2}\)

For small angles (\(\sin \theta \approx \theta\)): \(\frac{d^2 \theta}{dt^2} + \frac{g}{L} \theta = 0\)

This is a simple harmonic motion equation: \(\frac{d^2 \theta}{dt^2} + \omega^2 \theta = 0\) where \(\omega = \sqrt{\frac{g}{L}}\).

Angular Displacement

The general solution for the angular displacement \(\theta(t)\) is: \(\theta(t) = \theta_0 \cos\left( \sqrt{\frac{g}{L}} t + \phi \right)\) where:

• \(\theta_0\) is the maximum angular displacement (amplitude).
• \(\phi\) is the phase constant, determined by initial conditions.

Period of Oscillation

The period \(T\) of the pendulum is the time for one complete cycle of oscillation: \(T = 2\pi \sqrt{\frac{L}{g}}\)

overall 1:30

surprising, but with a deterministic chain of mathematical reasoning

Exceptionally high model fit

• real data from an actual student protocol (Milster, 2009):

33

Interpretation of Kuhn’s model

1 ungefähr 7min bis hierher

Shared symbolic generalizations

• e.g. \(F=ma\)
• abstractions that guide future research

1

Shared beliefs in particular models

• e.g. thermodynamics: gas molecules behave like tiny elastic balls in random motion
• such analogies guide future research

35

Shared values

• e.g. theories should be simple, logically consistent and compatible with other theories
• How much should we value replication?

40

Interpretation of Kuhn’s model

30

implicit processes: abmigious, not “we do not know”

Is Psychological Science Paradigmatic?

• How can we assess this? By examining the content and methods of our teaching.
• Books that students read and experiments they conduct.
• Is there a standardized approach to theory and experimentation?
• Does the field offer theoretical richness and analytical work?
• Are the experiments characterized by high model fit?

1:30

Formalisms

Myers famous introductory book to psychology contains 2 formulas and no symbolic exercises.

Formalisms 2

Goldstein’s famous introductory book to perception contains 4 formulas and no symbolic exercises.

Formalisms 3

Anderson’s famous introductory book to cognitive psychology contains 5 (relevant) formulas and 1 trivial symbolic exercise.

die letzten 3 folien: 1:40

Experimental Projects – Natural Sciences (Example: Physics)

• A set of universally acknowledged important experiments, such as Nobel laureate Robert Millikan’s classic oil drop experiment.
• Approximately 50 key experiments are recognized (Christ et al., 2021; Felgner et al., 2021).
• Most of these experiments, about 30, are conducted during undergraduate studies (Christ et al., 2021; Felgner et al., 2021).
• The goal is to absorb the foundational concepts of the discipline, develop a specific way of seeing, and become physicists.

1min

Experimental Projects – Psychology

• A large and varied set of experiments, primarily dependent on the subject area of instructors.
• At our institute (Institut für Psychologie, 2020, 2022), some of the topics explored include spatial orientation in virtual environments, time-to-arrival estimates in driving situations, interpersonal distance, the continued influence effect, the sense of agency in user experience, the influence of colors in app design, multistability (e.g., Necker cube), the privacy paradox, encoding specificity, memory-induced eye movements, mental workload, categorization and speech, and situation awareness.
• Only one experiment is conducted by each student.
• The goal is to learn statistics and experimental methods; the specific content of the experiment is almost irrelevant.

1min

The Pseudo-Paradigmatic Science

• Psychology is neither paradigmatic nor does it behave as if it were pre-paradigmatic.
• Pseudo-paradigm: a pre-paradigmatic science attempting to construct a disciplinary matrix without exemplars.
• This results in values that are not deeply internalized and a community that lacks cohesion.

1:10

Pseudo-paradigm: Psychological science

1:20

Revolution of values

It is not a revolution in the sense of the “cognitive revolution” or of a Kuhnian paradigm shift (Kuhn, 1962) because it is not about the content of our science. Rather, it is about the values we hold as we conceptualize, implement, analyze, and share our science.”

Spellman (2015), p. 886

1 ungefähr nochmal 10min bis hierhin

Example 1: Replication (1)

• In the past decade, there has been a surge of interest in replication.
• In paradigmatic sciences, education emphasizes replication, where students are trained to reproduce the most successful theories and experiments.
• In contrast, pre-paradigmatic sciences prioritize innovation, with subpar research often deemed unworthy of replication.
• Many current replication projects in psychology tend to be misguided, as they often focus on randomly selected experiments (Camerer et al., 2018; Open Science Collaboration, 2015) or prioritize studies that are easy and quick to run (Klein, 2014; Klein et al., 2018).

3min

Example 1: Replication (2)

• Arbitrary replication efforts may be considered a misuse of valuable resources.
• Stroebe (2019), p. 100: “However, the optimal strategy, and one that would result in the greatest gain in knowledge, would be to develop and test a better theory that has greater empirical content.”
• The primary aim of replication projects in psychology is to reinforce the importance of replication as a scientific value.

Example 2: Statistics

• Examples of proposed reforms include making power analysis mandatory, lowering the alpha threshold, and adopting Bayesian statistics.
• However, in paradigmatic sciences, statistics often plays a peripheral role. As Ernest Rutherford famously remarked, “If your experiment needs statistics, you ought to have done a better experiment.” (Ratcliffe, 2016)
• Statistical power does not influence effect sizes; even in a well-powered study, an effect size of \(r=0.1\) is not particularly compelling (consider the MAGIC criteria).
• Methodologists have yet to convincingly demonstrate that their advanced techniques lead to meaningful improvements.
• The ongoing debate about statistics primarily serves to reinforce the value of precise “mathematical modeling”.

3

Summary

• Without a theory of how science operates, any debate on progress in psychology is futile.
• Psychology lacks exemplars but behaves as it was paradigmatic, making it pseudoparadigmatic.
• Explicit debates shape the disciplinary matrix, primarily by highlighting the values psychologists prioritize.
• However, this approach hinders progress because it overlooks the priority of exemplars.
• The matrix is artificial, the values are not genuine, and the community is fragmented rather than unified.
• While there is no clear solution, the first step is acknowledging that psychology is not a paradigmatic science.

References

Camerer, C. F., Dreber, A., Holzmeister, F., Ho, T.-H., Huber, J., Johannesson, M., Kirchler, M., Nave, G., Nosek, B. A., & Pfeiffer, T. (2018). Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015. Nature Human Behaviour, 2(9), 637–644. https://doi.org/10/gd3v2n

Christ, A., Felgner, K.-H., Grätz, H., Jenderka, K.-H., Klemenz, A., Leschhorn, J., Reinshaus, P., & Stölzer, M. (2021). Grundpraktikum Physi III und IV (18th ed.). Martin-Luther-Universität Halle-Wittenberg Institut für Physik. http://www.physik.uni-halle.de/Praktikum/gp/anleitungen/Phy2-Heft-21.pdf

Felgner, K.-H., Fränzel, W., Grätz, H., Leschhorn, J., & Stölzer, M. (2021). Grundpraktikum Physik I und II (19th ed.). Martin-Luther-Universität Halle-Wittenberg Institut für Physik. http://www.physik.uni-halle.de/Praktikum/gp/anleitungen/Phy1-Heft-21.pdf

Institut für Psychologie. (2020). EmpEx 29. Abschlussveranstaltung.

Institut für Psychologie. (2022). EmpEx 33. Abschlussveranstaltung.

Klein, R. A. (2014). Investigating variation in replicability: A “many labs” replication project. Social Psychology, 45(3), 142. https://doi.org/10/f56ddm

Klein, R. A., Vianello, M., Hasselman, F., Adams, B. G., Adams, R. B., Alper, S., Aveyard, M., Axt, J. R., Babalola, M. T., Bahník, Š., Batra, R., Berkics, M., Bernstein, M. J., Berry, D. R., Bialobrzeska, O., Binan, E. D., Bocian, K., Brandt, M. J., Busching, R., … Nosek, B. A. (2018). Many labs 2: Investigating variation in replicability across samples and settings. Advances in Methods and Practices in Psychological Science, 1(4), 443–490. https://doi.org/10/gfsc2s

Kuhn, T. S. (2012–1962). The structure of scientific revolutions (4th ed.). University of Chicago Press.

Milster, S. (2009). Versuchsprotokoll zum Fadenpendel. https://people.physik.hu-berlin.de/~milster/F301.pdf

Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349, aac4716. https://doi.org/10.1126/science.aac4716

Ratcliffe, S. (2016). Ernest rutherford. Oxford University Press. https://doi.org/10.1093/acref/9780191826719.013.q-oro-ed4-00009051

Spellman, B. A. (2015). A Short (Personal) Future History of Revolution 2.0. Perspectives on Psychological Science, 10(6), 886–899. https://doi.org/10.1177/1745691615609918

Stroebe, W. (2019). What can we learn from many labs replications? Basic and Applied Social Psychology, 41(2), 91–103. https://doi.org/10/ggn9sv