Q: Why are there so few post-publication comments in PLoS ONE?

PLoS ONE is a lower-tier journal. Most scientists have a backlog of potentially highly important papers to catch up on. It is, thus, to be expected that most papers will get few reviews.

If we ever get the ultimate solution, i.e. the ideal free publication system with open post-publication peer review, most papers will get few reviews just as is the case now for PLoS ONE.

On the one hand, this serves a positive function: The community allocates its resources so as to pay more attention to papers with evidence of high quality. On the other hand, there is the danger that good or great work goes completely ignored.

To avoid the zero-reviews scenario, the traditional editor-soliciting of reviews remains an important mechanism in the new system: It helps get a basic quality estimate for each serious scientific paper, so as to get the broader reception of the paper on its way.

An additional mechanism for the new system is that of author-solicited reviews. This would work similarly to recommendations and could help extremely controversial work to acquire some support. Solicited reviews will need to be marked as author-solicited or editor-solicited, so this information can be taken into account by any automatic paper assessment function.

Open post-publication peer review is distinct from the PLoS ONE system. PLoS ONE combines traditional pre-publication review with post-publication commenting. While it promotes publishing of the pre-publication reviews alongside the paper, it allows reviewers to opt for their reviews to remain secret.

In open post-publication peer review, every review is public, including its numerical ratings. In such a system, each PLoS ONE paper would have two to three editor-solicited reviews at its side (PLoS ONE states that the average number of pre-publication reviews is 2.6) and the papers could be ranked according to their ratings.

In addition, the reviews would assess and rate importance along with technical soundness, whereas in PLoS ONE the reviews assess only technical soundness.

Technical soundness is a low bar, so acceptance in PLoS ONE will not place a random paper high on anyone’s list of reading priorities. The best papers in PLoS ONE probably deserve continued evaluation through peer review. The editor-solited review process should therefore be used to get initial numerical estimates of paper quality on multiple scales including importance. This would allow PLoS ONE fans to prioritize their reading of the journal and comment on the best papers.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: