Two verdicts to help peer review: reject-and-resubmit, reject-because-no-referee
Everybody knows peer review has its problems, but not much is done to try to fix them. In this post I briefly outline two relatively easy to implement changes editors could make to improve the efficiency of the system. The former is already in use at the philosophy journal Inquiry (and perhaps elsewhere; I make no claim to originality), and while the latter isn’t, at least part of the reason for that is that I just thought of it this morning.
Here’s a big problem with peer review: unshared knowledge. When a paper comes in, its editor doesn’t know its past: if it’s been reviewed before, where and by whom, and what the referees thought of it.
In part, this is arguably fair enough: it’s good for the paper to get judged afresh on its merits, and this information might bias the chance of this.
But it leads to great inefficiencies. One salient problem is that the editor runs the risk of sending the paper out to a referee (call them r) who has already seen and rejected it. This is a big time-suck: it requires the editor first to work out that r is suitable, to send the email inviting them, then to wait for the response. Sometimes these responses take a while, or don’t come at all (not frequently: the vast majority of referees and would-be referees are extremely helpful, in my experience). If the response does come back that r is recusing themselves because they’ve seen it before, it will take a while for the editor — who has other things to do — to get around to reading the response, and then researching a new name. Overall, this could take a week; if it happens four times, we’re taking a month, and that’s a decent chunk of time.
The problem is that past verdicts on the paper aren’t shared from journal to journal. Let’s assume that no centralized verdict-sharing mechanism is going to arise anytime soon (for one, it would be hard to implement in the different online submission systems). Is there any way for an editor to get knowledge of past verdicts?
Well, if they were responsible for the past verdict, the answer is obviously yes. If they selected the referee and made the decision, they would know at least which people they had asked to referee the paper and so, were they to find a new referee for it, would know at least who not to ask.
So this is my first suggestion: when a paper is promising but gets a reject verdict, reject it, but offer the author a chance to submit it again to the journal in question, with the understanding that it will be treated as a completely fresh submission. The editor can then make use of the labour they expended finding a reviewer for the first verdict: they can make sure not to ask those people again. This will lead to the second submission of the paper being more efficient than it would have been at another journal, because there’s a decent chance that the editor of this putative other journal would ask the people who decided against it at the original journal.
My first suggestion is essentially that papers stay with journals longer: potentially through several rounds of rejection and fresh submission. (You might think that no editor would go with this: they’re already swamped with papers. But if it were widely adopted, then they would be less swamped. Of course, this is subject to a free-rider problem.)
My second suggestion is kind of the opposite: that one reject possibly perfectly good papers — papers one wouldn’t desk reject — because one fails to quickly find a referee for them.
Here’s the way I think of it: most editors have two main ways of finding referees: certain algorithmic procedures probably shared by all other editors, and their own personal connections, which probably aren’t shared by all other editors. Examples of shared algorithmic procedures include: checking the bibliography, checking google scholar citations for papers discussed in the paper, and (for philosophy) checking philpapers for recent papers published in the area. Personal connections include friends, colleagues, visiting speakers, social media connections, graduate students, etc.
In my experience, if both the algorithmic procedures fail, and I don’t know anyone, it’s likely to take a disproportionate amount of time for me to find a referee. It’s doable, but it’ll require a decent amount of research: several pages deep in google scholar, skimming the SEP (an online philosophy encyclopedia), etc.
Based on this, the thought is: if the two main ways fail, some other editor might have better luck, in particular by virtue of their different personal connections. So, I’m tempted to say that, to reduce the amount of time it takes for a paper to find a reviewer, if an editor fails at first, they should take a pass, reject the paper because they can’t quickly find a reviewer for it, and let someone else have a go. And that’s the second suggestion.
These two suggestions perhaps point in different directions, and the second, as the first, will suffer from free-rider issues. Work and thought are needed to improve on them. But this work and thought is surely worth doing, given the importance of the peer review system for all of us, and its problems.
