Great to see this here. I would like to see more vocal and informed criticism of the current status quo on our peer review institutions. It's more than a little absurd that people seem to conflate the present means of implementing peer review with the notion of peers reviewing one another more broadly, as though objecting to the current way of doing things is a repudiation of the very notion of experts in a field reviewing one another's work. It isn't. What's also a bit odd is how little in the way of positive arguments and evidence there are in favor of the present system, relative to how aggressively people either defend it or ignore its shortcomings. The bizarre lack of transparency in the way most peer review is conducted alone should be a huge problem.
I wonder how many of our modern social problems were caused by bad social science research and the subsequent policy choices made based on that research.
I work regularly with historical literature on topics in ecology, biology, physics and economics. It's very difficult to make broad statements since each discipline or field has its own practices of writing and publishing. These practices are confounded by the epistemic development of study designs and frameworks. For example, old articles in Ecology (say from the 1900s) are challenging to exploit, compared with articles (of the same age) in Physics or Chemistry.
What I see is that truisms and stereotypes stick around for decades without ever being tested. I'm not referring to repeating experiments, but the theoretical frameworks and assumptions underlying research areas. This affects all parts of the process, including what (and who) is considered "publishable" at different times and places. And all of this is conflated with modern 'publish or perish' pressures and the article economics operating within institutions which affect funding, promotions and visibility of certain results.
I find the current very open publishing paradigm incredibly exciting. Peer review is just one part of the process of getting our ideas and work out there to be discussed. It's not perfect and passing peer review should never be understood as representing an absolute test of truthfulness. As you demonstrate, there are other options. Let's celebrate those.
Great to see this here. I would like to see more vocal and informed criticism of the current status quo on our peer review institutions. It's more than a little absurd that people seem to conflate the present means of implementing peer review with the notion of peers reviewing one another more broadly, as though objecting to the current way of doing things is a repudiation of the very notion of experts in a field reviewing one another's work. It isn't. What's also a bit odd is how little in the way of positive arguments and evidence there are in favor of the present system, relative to how aggressively people either defend it or ignore its shortcomings. The bizarre lack of transparency in the way most peer review is conducted alone should be a huge problem.
I wonder how many of our modern social problems were caused by bad social science research and the subsequent policy choices made based on that research.
I work regularly with historical literature on topics in ecology, biology, physics and economics. It's very difficult to make broad statements since each discipline or field has its own practices of writing and publishing. These practices are confounded by the epistemic development of study designs and frameworks. For example, old articles in Ecology (say from the 1900s) are challenging to exploit, compared with articles (of the same age) in Physics or Chemistry.
What I see is that truisms and stereotypes stick around for decades without ever being tested. I'm not referring to repeating experiments, but the theoretical frameworks and assumptions underlying research areas. This affects all parts of the process, including what (and who) is considered "publishable" at different times and places. And all of this is conflated with modern 'publish or perish' pressures and the article economics operating within institutions which affect funding, promotions and visibility of certain results.
I find the current very open publishing paradigm incredibly exciting. Peer review is just one part of the process of getting our ideas and work out there to be discussed. It's not perfect and passing peer review should never be understood as representing an absolute test of truthfulness. As you demonstrate, there are other options. Let's celebrate those.
Fantastic piece