How do I stay fair when reviewing a manuscript?


Column by Marie Chancel, post doc at Karolinska Institutet, about her principles as a reviewer.

Sometimes, I’m a reviewer.

But I also write scientific articles myself, and I sometimes perceive the reviewing process as a gigantic loss of time instead of a process to make research better. Yet, I’ve witnessed awesome supervisors, collaborators, and mentors for whom the ’scientific community’ was not a vain word, who will not count their hours to help other researchers without anything in return. Having people like them assessing soon-to-be-published manuscripts would only mean to improve these said manuscripts. So how to contribute to a welcoming community of researchers to help produce better work, without perpetuating unnecessary and detrimental attitudes that turn peer-reviewing into an unfair, hurtful system?

I’ve witnessed awesome supervisors, collaborators, and mentors for whom the ’scientific community’ was not a vain word…

Recently I’ve read a lot of testimonies from researchers raising the same question. Some people choose to stop reviewing, to publish only preprints, to ask for compensation when reviewing. There’s probably a lot of other initiatives or self-applied rules that we don’t know about. I’ve personally decided to follow some principles based on my own instinct, my experience as an author and mostly some great insights collected here and there among researchers I respect and admire. Easy to implement, they make me feel okay with myself as a reviewer, even if sometimes it’s hard and scary to follow them.

1. Reviewing is opening a discussion: When I receive a paper to review, I want to have the same mind set as when a member of my lab asks for my advice on a project or a study. How can I use what I know to make this study better? Point out the great findings as well as the biggest limitations. If I disagree with a statement or an analysis, I will always consider that I may have misunderstood. If it is the case, I may not be the only one so how about rephrasing? If I understood right, it is my responsibility to justify why I think it is wrong and to propose a correction. I will always leave the possibility to the authors to respond and justify their choices. This means that unless I see a big problem in the way data have been acquired, I will never suggest rejection.

I will always leave the possibility to the authors to respond and justify their choices.

What is the point of rejection if the paper is just sent to another journal with none of the improvements I think would have been important? Did I “help science” then? I will make myself available to go to the end of the reviewing process, to contribute as much as I can to the improvement of the manuscript before publication. After a few reviews following this principle, I realized that an unexpected consequence is that I spend more time on a review. I think it is necessary and I also feel more comfortable with refusing to review if I don’t have the time to properly respect these guidelines.

2. I am responsible for my comments: I sign my reviews. Even when it is expected that I am anonymous. Even if I am terrified that the authors will judge me and think I am stupid and did not understand their work. Even when among the authors, there may be influential people in my field. I spend time on my review, I respect the authors, I try to be mindful of how I say things in addition of what I say. I am doing my best because I am not an enemy, I am a temporary collaborator trying to check if everything looks legit and solid. I can only wish this is visible in my comments, so I sign them. Wish me luck!

I sign my reviews. Even when it is expected that I am anonymous.

3. I am not taking into account the journal: Editors be warned, I don’t care for your impact factor. I do not believe impact factor is a good way to assess individual research, so I’m not going to judge if a study is worthy of something I don’t recognize (also, you don’t pay me, so I do what I want). This is controversial I imagine, because I can see some counter arguments. Journals cannot publish everything, of course, but that’s an editor’s problem. As a reviewer, my main concern is “is this good science?”. We are often asked to assess the novelty. Yet, more and more concerns are raised about the replication crisis. I choose to consider novelty as a good point but not above a solid method or a strong replication.

I don’t care for your impact factor.

Of course, not every work provides the same contribution to its field, not every finding is ground breaking, but that’s the beauty of it! We all know examples of awesome studies that are published “low”, i.e. in journal of lesser renown (check out my work obviously!) and we are aware of factors that can help publish “high”, i.e. in famous high-impact journals, that have nothing to do with rigorous scientific work (gender of the first authors, number of “high” references, the list can go on forever). How is it helpful to maintain a system I don’t believe in? I also like this principle because it keeps at bay the little voice that makes me feel envious or just sad every time I review a paper in a journal much higher than my usual publication level. I am really not proud of this, but I’m trying to be better and this principle helps.

My little principles are insignificant in regards to the global situation and issues with publication, grant and funding, positions availabilities, and other challenges. A whole new system needs to be proposed, but who’s got time for this when your contract/PhD/grant (delete where applicable) finishes at the end of the year?

Meanwhile, here are the principles I choose to follow. I’m sure they will evolve as I keep meeting incredible people along the way, and I am looking forward for this.

1 kommentar

Tack för din kommentar. Den kan komma att modereras innan den publiceras.

  • Dagmar Lagerberg

    Marie, Thank you for your brilliant article.