Over on his Experimental History blog, which he describes as “cognitive sneezing and interior design for your head“, Adam Mastroianni has two great posts about peer review in science. Wikipedia defines peer review as “the evaluation of work by one or more people with similar competencies as the producers of the work (peers). It functions as a form of self-regulation by qualified members of a profession within the relevant field.”
Mastroianni explains that although ubiquitous today, peer review is a relatively new phenomenon. After World War II governments poured huge amounts of money into research; peer review was supposed to make sure the money was well spent. But as Mastroianni documents, peer review has failed on just about every metric.
Research productivity has been flat or declining for decades; reviewers consistently miss major flaws in submitted papers; fraudulent work is published all the time. Peer review often encourages bad research because of unhelpful comments; and scientists themselves don’t care about peer review: they actively seek to circumvent it, and ignore it in their own reading.
In the face of this undeniable failure, Mastroianni goes on to consider whether it can be saved. He discusses the obvious approaches that might be applied to do that, and convincingly shows that none of them will work. Peer review has not only failed, it’s unfixable. Mastroianni is “relieved”:
Waiting months just to hear that an editor didn’t think your paper deserved to be reviewed? Reading long walls of text from reviewers who for some reason thought your paper was the source of all evil in the universe? Spending a whole day emailing a journal begging them to let you use the word “years” instead of always abbreviating it to “y” for no reason (this literally happened to me)? We never have to do any of that ever again.
He’s relieved, too, because jettisoning peer review and its constraints has given him a new freedom – and a new pleasure in writing papers that people will want to read:
last month I published a paper, by which I mean I uploaded a PDF to the internet. I wrote it in normal language so anyone could understand it. I held nothing back—I even admitted that I forgot why I ran one of the studies. I put jokes in it because nobody could tell me not to. I uploaded all the materials, data, and code where everybody could see them. I figured I’d look like a total dummy and nobody would pay any attention, but at least I was having fun and doing what I thought was right.
Then, before I even told anyone about the paper, thousands of people found it, commented on it, and retweeted it.
What Mastroianni describes is essentially the diamond open access approach, something Walled Culture has discussed several times. It is designed to provide an extremely simple and lightweight publishing platform to help researchers get their papers quickly and easily before as many people as possible. It is costless, for both the person uploading their paper, and those who download it.
One of the strongest and most frequent arguments against the minimalist approach of diamond open access is that it often comes without the traditional apparatus of peer review, organising which would be too costly for what are typically tiny publishing budgets. Mastroianni’s posts confirm its absence really doesn’t matter, because peer review adds nothing, and may even reduce the scientific value of papers that undergo it. That’s great news for open access, whose gold and green forms are still struggling to achieve their aims. It’s also great news for science and thus society, both of which stand to gain easier access to more interesting and potentially more useful research.
Featured image created with Stable Diffusion.