Two views on peer review

Tagged , ,

10 thoughts on “Two views on peer review

  1. Rob H. says:

    I recently came across this comment in the notices by Brian Osserman (http://www.ams.org/notices/201210/rtx121001383p.pdf). It suggests an official two-step process: first, figure out if the paper is good enough for the journal, then make sure it’s correct. These steps would be carried out by two different people with the possibility that the second person is not anonymous. Interesting.

  2. @Rob: I believe a number of journals already do approximately what Osserman suggests. I get a fair number of requests for “quick opinions” on papers where I’m only asked to judge whether the paper is likely meet the journal’s quality standards were it to be fully refereed. In such cases, my impression is that the editor is soliciting several such options and will only send it out for a full refereeing if most/all of the quick opinions are positive. The top general and specialty journals do this the most (and some have done so for at least a decade) but it seems like the practice is expanding. I don’t know if it actually helps speed up the rejection process…

  3. Rob H. says:

    @Nathan: Indeed, Osserman mentions this, but wants to take it a step further. For instance, I have recently had a paper rejected where, after about a month, I received a thoughtful email from the editor explaining how several experts felt about the paper. I have also had a paper rejected where, after about 6 months, I was forwarded a one paragraph review from a referee that indicated the referee must have read at most half of our paper. Osserman’s suggestion aims to eliminate the latter result.

  4. xi'an says:

    Nicolas Chopin, Andrew Gelman, Kerrie Mengersen and myself (all statisticians) wrote a more moderate perspective on peer review: it got rejected by several journals and eventually made its way to the ISBA Bulletin.

  5. @Rob: Thanks for the clarification, of course I didn’t actually read Osserman’s article. Having done so now, I’m not sure how practical his idea is of having the journal make a binding decision about acceptance before the full refereeing (acceptance being contingent on the referee finding that the paper is correct). Sometimes the quality (or lack thereof) of a paper is more determined by the details of the proofs rather than just the statement of results. Or perhaps the paper is part of a series and it’s not clear at first glance if this is the key paper or just a technical follow-on.

    A different approach to speeding things up would be to have hierarchies of journals controlled by a single editorial board. You would submit your paper to the editorial board and they would get both quick opinions and a full report and then decide which journal it was worthy of appearing in. The range of quality of the constituent journals would need to be broad enough that you are basically guaranteed acceptance in one of them if the paper is correct. One could even copy the CS folks and have an editor, the quick opinionators, and the referee engage in an online discussion about the placement after all the data is in…

  6. Rob H. says:

    @Nathan: I think you’re on to something with your idea. One problem though is that sometimes one is rejected with the reason “This paper should be in a more specialized journal”. I do however have a twist on your idea that I think would be easier to implement. Here’s something that has happened to me: the referee likes the paper, but thinks it should be in a less prestigious journal. The editor asks the referee for suggestions and asks if the referee is willing to continue on with the other journal. I receive an email from the editor saying that the referee would be happy to produce a new, favourable report for acceptance in such and such journals. So, then I contact an editor at one of those journals and everything works out. I wonder whether one can encourage editors to ask such questions to the referees. As for the referees accepting to do this, that could end up being part of the culture of refereeing (not having to do it each time, but understanding that it should happen at least sometimes).

  7. At least one journal actually now mentions in its refereeing requests that, in addition to considering the paper submitted for Journal X, the referee might consider its suitability for slightly less prestigious Journal Y if it does not quite seem to reach the level expected from X (Journal Y is edited by the same publisher, in the case I know, which I guess makes this arrangement easier to organize).

  8. Tom Leinster says:

    Emmanuel, is that arrangement official and publicized? Your “X”s and “Y”s suggest that it’s not, but that seems like a missed opportunity: authors would surely be more likely to submit to Journal X if they knew they did this. (On the other hand, maybe that’s what Journal X is trying to avoid.)

    Are you willing to say what the values of X and Y are?

  9. Sorry, I didn’t check the comments since I left the message. I don’t think the arrangement can be considered a secret (since any reviewer will have heard about it since it began), but I checked on the web site of the publisher (an academic organization), and it is not mentioned there, so I’m not sure exactly what to do… Let me say that X is an old and venerable number theory journal, and that Y is less well-known, but edited by the same academic publisher.

  10. This is not mathematics, but American Physical Society does submission-forwarding. Physical Review Letters (PRL) is their flagship journal that covers all branches of physics, but they also publish specialized journals (Physical Review A, B, C, D, and E). Referees are asked if manuscripts fit those specialized ones if they’re not of PRL quality. So, for example, if your paper gets rejected by PRL for the very common reason “awesome but not awesome enough for PRL,” you’ll be contacted by one of the specialized journals if you’d like your paper to be considered for publication in it. If you say yes, referee reports will be forwarded to the chosen specialized journal, and the editor of this journal decides if another round of review is necessary.

    Nature and its sister journals also do this. And this is not secrete at all; it’s explicitly written on those journals’ websites. Nature, PRL, and the like are on the extreme end of prestige, and they get a ton of submissions each day. But they’re operating just fine with their submission forwarding policy. So, it doesn’t seem like the alleged increase of submissions is bogging down their systems.

    Then again, those journals employ full-time editors for rapid publication. And I’ve never heard of a math journal with a bunch of full-time editors who make final decisions. So, simply adopting the forwarding policy may not work as well in mathematics. But I’m curious to see what happens if AMS does the same as American Physical Society, so a paper rejected by Journal of AMS may be transferred to Transactions etc. with referee reports.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 554 other followers

%d bloggers like this: