Users can point out weaknesses in scientific articles on the Pub Peer forum. It could include image manipulation or errors in statistical analysis. The picture shows spores formed in the fungus' sporangia.

Forum for criticism of published articles

Svenska 2016-01-21

There is a great pressure to publish as many items as possible in the most highly ranked journals. Sometimes the process is too fast. Forums such as Retraction Watch and Pub Peer make it possible to criticise what has already been published.

Pub Peer is an Internet forum that discusses already published articles. The site has received a lot of attention since it has drawn attention to several articles believed to be based on obvious carelessness or even fraud. One of the members who commented on Pub Peer has been sued for slander.

Boris Barbour is a researcher at the Ecole Normale Supérieure in Paris, and is the treasurer of the foundation behind Pub Peer. Internet forums like Pub Peer, according to him, are necessary to allow important criticism of published articles to rapidly reach those who use the research – such as other researchers, investors and care planners.

“Incorrect results undermine the work of other researchers who base their research on them. They waste taxpayers’ money and could even cost lives, if medical guidelines are based on incorrect research. Traditional methods of discussing scientific articles are just not efficient enough.”

No systematic evaluation

There is no systematic evaluation of articles at Pub Peer; those users who want to can comment on an article. Users can point out any weaknesses they discover.

“It could be a question of image manipulation in gel or microscope images that had not previously been noticed. Other issues are errors in experimental design, statistical analysis or how the results are interpreted.”

One rule is that the criticism must be based on facts which can be verified – usually in published data.

“It is important in order to avoid slander. For example, discussing the intentions of a researcher is irrelevant.”

Anonymous comments allowed

Comments from registered users are published directly without being checked. Users can report what they consider is breaking the rules. Comments from unregistered users are checked first to ensure that they comply with the rules. One can remain anonymous.

“Even though it would be better if everyone felt safe enough to criticize others’ articles openly, it is too risky for their careers to do that. That is not likely to change in the near future.”

Reactions to the site have been varied. There are those who give it strong support and others who do not like the site. According to Boris Barbour, however, Pub Peer is currently one of the most popular forums for the scientific criticism of articles.

“Pub Med Commons only covers biomedicine and does not allow anonymous comments.”

Articles that are retracted

Ivan Oransky is a science journalist who thought that articles that had been retracted could very often be used as if they were still bona fide. The process of retracting articles often passes unnoticed, possibly because it happens rather a long time after publication.

It is often unclear why articles are retracted. That was why he and Adam Marcus started a blog that reports on them. They find the retracted articles through tips or the article database, Pub Med.

“We contact authors and journals and ask what happened.”

They report mainly in the field of biomedicine. That is where there are most articles – and the highest number of retractions.

Not only is the number of retractions of articles increasing, but also the proportion.

“There were ten times more in 2011 than in 2001, according to statistics published in 2011 by the journal Nature. In the same period, the number of publications increased by 44 per cent.”

Good reception

Retraction Watch does not carry out scientific reviews of articles. They just report on articles that have been withdrawn. However, Ivan Oransky wanted to create a forum where visitors to the blog could discuss published articles.

The blog has been given a good reception.

“At first people were sceptical. They didn’t know who we were. We now have 11,000 subscribers and we have received funding from the MacArthur Foundation.”

Ivan Oransky has a clear idea of what lies behind the growing problems in scientific publishing.

“The problem is the incentive. Publication is all that counts. People take short-cuts to get their writing published. This pressure on researchers creates the conditions for the science we get.”

Also read in Curie: Ranking overrated in current publishing

Text: Anja Castensson
Photo: Science Photo Library / IBL Bildbyrå