UPDATE: I found this video of @dwhly on the THL News Blog and recommend it for your next free 63 minutes.

Peer review gives sense to publications by “affirming” their scientifical relevance and qualtiy. Recently the publishing company Elsevier launched a competition for “original idea[s] (…) that will significantly improve or add to peer review as it exists in its various forms for academic journals” with finalist to be announced in mid-august. Reffering to this challenge James Rosindell and Will Pearse presented a peer review model on PLOS Biology that very much resembles open peer review proceedings (OPR) and added some interesting thoughts. I was thinking about writing a critique on peer review from a radical democratic perspective. As some of my ideas matched up with the proposal of Rosindell and Pearse I want to discuss them shortly along their approach.

In short, the idea is to establish an online system in which commisioned scholars but also other volunteer can evaluate papers with a system comparable to Reddit. This way the authors intend to open the arena where science is produced, while keeping the review´s filtering function for a wider range of readers.
Breaking the consequences down:

1. The process of filtering becomes visible to the scientific community, while today it is in most cases locked up in publishing houses.
2. The review process becomes part of the debate.
3. Generally, more and diverse actors are able to take part in the review process.
4. Unpublished manuscripts would remain “in the orbit” and could be published later in case of rejection.

The decisive difference to current OPR might be the intention to head for a broad participation within the scientific community, by making it easier to give a vote. Recent OPR proceedings – at least as far as I see didn´t manage to massively activate co-readers, probably also caused by non-intuitive, sometimes old-fashioned systems. The proposal would  allow in perspective the sepparation of professional review from publishing houses, because papers could be published on the platform independently on whether there is a call or not. As papers stay on the platform and can be published at a later point, the history of publications documented on the platform could also be read as a critique of prevailing knowledge structures.

While I generally agree with open peer review proceedings and the proposals made by Rosindell and Pearse, here are some of the problems I see:

1. General overviews are a handy thing, but they don´t correspond to the sophisticated structure of scientific texts (there might be differences between disciplines). A comparable problem yet regards comments and reviews when summarizing or valuating the text in a rather general way. In this case the critique on specific aspects is not comprehensible to the reader.
2. Overall-ratings suggest the validity of the scientific publications. However, looking back in time, we have to bear in mind that overall convictions of the past in many cases prevented a proper (from the perspective of the prevailing present) change of theory, methods or perspective (for examples of prominent peer-review-victims click here). Insofar I don´t agree with the idea of an advancing science – especially with the number of publications as index. Discoursive structures arise by producing a supposedly “general” opinion. It is important to enable the reader to construct an opinion on their own.
3. Rosindell and Pearse propose the exclusion of participants that are considered to be “biased” or just “not qualified”. But who defines quality?

Propositions

1. The platform is replenished with a social network of (dis)trust. The overall rating of papers change depending on the trust attributed to the people within the network.. This doesn´t change the problem of arising discourses, however, it visualizes discoursive structures and thus makes them more permeable & open to criticism.
2. The network of (dis)trust should not only relate to papers and reviews but also to parts of the text. Besides writing a full review, participants should be able to write comments and opinions within the paper (like in Google Docs, Zoho, aso). Full reviews can relate to these elements. This way review and text form a unit, which gives way to a scientific debate along the text.
3. In a system of (dis)trust there is no need for restriction as the restrictions are being made by the users themselves. Again: that doesn´t solve the problem of exclusivity but it empowers the users to do it on their own. This way people can get an idea of how their scientific peer group changes their reception of science.