Back in 2008, Walter Mischel wrote the following for his column in the APS Observer:
Years ago, a wit (I have forgotten his name) called it the toothbrush problem: Psychologists treat other peoples’ theories like toothbrushes — no self-respecting person wants to use anyone else’s. It’s amusing, but it also points to a conflict that we may be nurturing within our profession to the detriment of our science.
It's worth going back and reading these moments of self-reflection, composed in the relative calm before prominent failures to replicate began to accumulate. Mischel is voicing a concern, but is hardly raising an alarm bell. To him, the fact that (a) every tenured faculty members has their own toothbrush, and (b) these appear to coexist in parallel is not a sign that the science being done is fundamentally flawed. Rather, he writes that it's a sign that the pressures of publishing and tenure are making the field, at worst, stressful and inefficient.
Now, compare Mischel's prognosis with this account of psychological science, published by Paul Meehl in 1990.
Null hypothesis testing of correlational predictions from weak substantive theories in soft psychology is subject to the influence of ten obfuscating factors whose effects are usually (1) sizeable, (2) opposed, (3) variable, and (4) unknown. The net epistemic effect of these ten obfuscating influences is that the usual research literature review is well nigh uninterpretable. Major changes in graduate education, conduct of research, and editorial policy are proposed.
It's very much worth your time to read Meehl's article in its entirety, and to do so with a self-critical frame of mind. For a long time, psychology has been a field in which "agree to disagree" has been an acceptable position for everyone to adopt, from graduate students to faculty to editors. The hard truth, however, is that a field in which everyone can agree to disagree has no substantive criterion for deciding what is and is not true. Circa 2008, the status quo was one in which most researchers were content to live in collective wonderment at the possibilities of their theories, without calling one another to task on what, exactly, those theories entailed or implied, or whether those theories were internally consistent, or whether those implications appeared to be at odds with the implications of other theories. It's not that disagreement never occurs, but that a priority has been given to peaceful cohabitation within the field, even in cases where some of us have been walking around with highly suspect toothbrushes.
(Continue reading)