This post is part of an online article I wrote for Publons. Publons is an online management system where you can keep track of your peer reviews, and compare your records to others. I wrote the article as a helpful guide for beginning peer reviewers. The article consists of four parts:

1. How to critically read a manuscript
2. Writing the peer review
3. Submitting the peer review
4. Managing peer review requests

Peer review is a great way to become a better scientist. By regularly doing peer reviews, you will start to think like the peer reviewer who will criticize your paper, so it will make you a better writer. It is an essential part of being a scientist. And, to be honest: it can feel really great to be in the power seat sometimes! But, remember to always stay friendly and polite.

4. Managing peer review requests

Which and how many peer reviews should you perform? Most junior scientists will start to get peer review requests after publishing their first paper as a first author. Often, you will get asked if one of your papers is listed in the references of the manuscript, so the new study will be in your field. You should only accept the peer review if you feel you have expertise in the topic of the paper, even if you are not an expert in all the techniques used. Typically, a manuscript will be sent out to about 3 reviewers, so as a rule-of –thumb you should perform 3 times more reviews than the amount of manuscripts you typically submit per year. More is better! Once you have peer-reviewed for a journal, the journal will ask you again, but usually only once or twice a year. The more papers you have published, the more requests you will get. You do not have to accept all of them, though. I’ll try to only have 2 ongoing peer reviews at the time; if I get more requests, I will turn them down until I have finished the previous ones. However, I will always accept requests to re-review a manuscript for the second time, and give priority to requests from journals where I serve in the editorial board.

Publons has been a great resource for me to keep track of all my peer reviews. It is a nice way to get recognition for all the work we peer reviewers do, mostly anonymously. In addition, it is great to compare my acceptance rate and length of peer reviews to that of others, and to have a feel for how many reviews other scientists perform, and for which journals. I can highly recommend signing up!

[hr]

This post is part of an online article I wrote for Publons. Publons is an online management system where you can keep track of your peer reviews, and compare your records to others. I wrote the article as a helpful guide for beginning peer reviewers. The article consists of four parts:

1. How to critically read a manuscript
2. Writing the peer review
3. Submitting the peer review
4. Managing peer review requests

3. Submitting the peer review

Once you have written your review, you will have to upload it into the journal’s reviewer interface. Most of them will have a box where you can assign the paper to one of 4 categories: accept without edits (only to be selected if you reviewed The Perfect Paper!), accept with minor edits (addressing typos, unclear sentences, or a small figure edit), accept with major edits (addressing bigger issues such as changes to introduction scope, interpretation of results, additional graphs or analyses) or reject (if the manuscript was not novel at all, not suitable for the scope of that journal, or contains plagiarism or other questionable practices). Most of the papers I have reviewed were classified as “accept with major edits”; I have selected the “reject” category less often. On the journal’s website, there is usually also a box where you can give specific comments to the editor; these will not be forwarded to the authors. Here you can state your personal opinion, or any issues you do not want to share with the authors. Your peer review will go into the section labeled “comments to the authors” – often by simply copy/pasting it into the appropriate box. Even if I think a paper should be rejected, I like to share my thoughts about the manuscript here, so that the authors can improve their paper before submitting it somewhere else.

[hr]

This post is part of an online article I wrote for Publons. Publons is an online management system where you can keep track of your peer reviews, and compare your records to others. I wrote the article as a helpful guide for beginning peer reviewers. The article consists of four parts:

1. How to critically read a manuscript
2. Writing the peer review
3. Submitting the peer review
4. Managing peer review requests

2. Writing the peer review

As I am reading the manuscript for the first time, I will have a text editor open in which I immediately write down small comments on specific parts of the manuscript, such as a typo in line 15 or an unclear sentence in the introduction. While I go through the paper, I will start to write down more general thoughts as well, such as remarks about the length of the introduction or a misinterpretation of results. After reading the whole paper, I will then re-read the abstract to see if it correctly captured hypothesis, experiments, results and interpretation. At the end of my read-through, I try to structure my peer review into three parts.

• Summary: A couple of sentences describing what the authors did, in my own words. This is especially helpful to refresh your memory when you will be asked to do another round of review on a paper you already reviewed before. Here, I will also give a general opinion about the paper, without mentioning if I think it should be rejected or accepted with edits.
• General comments: Some broad thoughts about the paper, such as: novelty of the findings, length of discussion, order of results, any concerns about data analysis or interpretation, language issues, etc. I will usually have about 3-5 numbered bullet points here.
• Specific comments. A numbered list of issues that refer to a very specific portion of the text or figures. Here, I might point out typos, missing definitions or abbreviations, unclear sentences, a missing reference, or suggestions to improve a figure or table. Usually, this part of my review will have about 20 remarks (but sometimes much more!). Even if this part is a long list, most of these points should be very easy to address by the authors.

It is important to number your remarks, making it easier for the authors to respond to each one of them.

In your review, the most important thing to keep in mind is to remain friendly and reasonable. You should feel no regret publishing your review under your full name. On the other hand, you do have the right to ask the authors to make primary data publicly available, perform some small and easy additional experiments or analysis, or change the layout and order of their graphs. Depending on the scope of the journal, it is however not reasonable in most cases to ask the authors to do large amounts of additional work. If you think the science is good, it should be published. There is always a need for additional experiments, but that can be put into another paper.

[hr]

This post is part of an online article I wrote for Publons. Publons is an online peer review management system where you can keep track of your peer reviews, and compare your records to others. I wrote the article as a helpful guide for beginning peer reviewers. The article consists of four parts:

1. How to critically read a manuscript
2. Writing the peer review
3. Submitting the peer review
4. Managing peer review requests

Peer review is an essential part of science. It is the part of the scientific process where our peers will have a chance to review our work, check it, comment on it, and – most importantly – determine whether it’s good enough to become a permanent part of the scientific legacy. At the same time, it is also one of the most dreaded parts of science, both for authors, whose work will be scrutinized or could be rejected by competitors, as well as for reviewers, whose inboxes are filled with a never-ending stream of peer review requests.

How to critically read a manuscript

So how do you write a good peer review? To help the inexperienced peer-reviewer, I’ve made a list of general questions to ask when you are reading the paper. Asking these questions should help you form an opinion about the paper, even if you have no idea where to start. It’s the list that I wished I had access to when I started my first peer review. Here we go:

• Do you have a conflict of interest when reviewing this paper? Do you collaborate with these authors, are they your personal friends, or are they direct competitors? Have you reviewed (and rejected) this paper before? If so, you need to decline this peer review and let the editors know.
• Do the title and abstract cover the main aspects of the work, would it spark interest to the right audience?
• Is the Introduction easy to follow for most readers of this particular journal? Does it cite the appropriate papers? Does it provide a hypothesis or aim of the study?
• Does the Methods section provide enough details for the general reader to repeat the experiments?
• If you skip the Methods, does the Results section give the right amount of detail to understand the basic details of the experiments?
• Does the Results refer to the figures in a logical order? Do the numbers in the tables add up correctly? Are any figures/tables mislabeled or unclear?
• Given the data that was obtained in this study, did the authors perform all the logical analyses? Did they include the proper controls?
• Does the Discussion address the main findings, and does it give proper recognition to similar work in this field?
• In general, is the paper easy to follow and does it have a logical flow? Are there any language issues?
• Did the authors make all their data (e.g. sequence reads, code, questionnaires used) available for the readers?
• Is this paper novel and an advancement of the field, or have other people done very similar work?
• Finally (and hopefully you will never have to answer yes to any of these question): Does the paper raise any ethical concerns? Any suspicion of plagiarism (text or experiments), duplicated or tampered images, lack of IRB approval, unethical animal experiments, or “dual use of research concern”?

[hr]

Publons logo

Since a couple of weeks, I have been an Advisor for Publons, an online community where you can get credit for pre-publication peer review, and get to choose which information you would like to share (e.g. displaying only the journals you reviewed for, not the actual papers). You can see my reviewer profile here. Because of this new role, I will post some thoughts on peer review here every now and then.

As a peer reviewer I like to do a good job in going over another scientist’s paper to make sure it will be a good addition to the ever-growing amount of scientific literature. I will check for the quality of writing, design, execution, and interpretation. But I will also look for small errors or omissions that might become a pain after publication, such as referring to the wrong figures, omission of method details, missing abbreviation explanation, or switched panels in graphs.

If you are the author of a new paper, and ready to send it out for peer review, you already know the requirements of the journal you are submitting your paper too (“Instructions for authors”). Here are a couple of items to check for that might not be on that list, but that will make the job of a peer reviewer much easier. It will also brighten their mood. The last person you want to review your paper is a reviewer who is in a bad mood.

• Line numbers: Even if the journal does not require this, please, please, please include line numbers in your manuscript. Continuous line numbers (as opposed to starting with 1 at every new page) are the best, if allowed by the journal. It is so much easier for me to refer to line 369, than to write “In the third paragraph on page 15, in the sentence that starts with “In addition”…”
• Check the figure numbering. Of course, errors will happen, and your PI asked you to add that one figure at the very last moment, but it is really nice if all referrals to figures are correct.
• Check the reference numbering. See Figure numbering.
• Methods – Pretend to be a person who is not familiar with your lab. Could you repeat the experiments roughly the same way as you did based in the information in the Methods? If not, is there additional information that you should share? Did you define all the buffers, manufacturers, PCR programs, number of subjects, samples, reads?
• Limit the referrals. In the methods, do you refer for something simple to another paper, and does that paper refer to yet another paper? Don’t make the reviewer or the reader look up something simple, such as a primer sequence or the name of a DNA extraction method in another paper. It’s fine to refer, but put the essential information in your manuscript.Don’t piss off the reviewer or the reader!
• Availability of raw data. The raw data should be available for the reviewer. Don’t make them have to ask you for the sequence or expression data; make them available in a public database that is accessible at the time of peer review.
• General order in graphs. If you have multiple groups of samples, please show them all in the same order in every graph, and choose a logical order. E.g. children on the left, adults on the right. Or, control on the left, treated group on the right. Oral on the left, gastric in the middle, and stool on the right.
• Order of the colors of bar/column graphs. Okay, this is one of my pet peeves that other people probably find very non-interesting, but I will include it here anyway. check if the legend/color key is in the same order as the in the graph.  In the example below, the graph has the light blue at the bottom, and the brown at the top, but in the key on the right, the order is the opposite. This makes these graphs harder to interpret. Figure taken from Vincent: A Python to Vega Translator: Charts Library. By the way, I really like this color scheme. Just not the order.

http://vincent.readthedocs.org/en/latest/charts_library.html

These are just some common issues that I often encounter during peer review (other than the bigger issues, of course, such as over-interpretation of data or missing control groups). Do other reviewers have other pet peeves with manuscripts that they want to share here? Please leave them in the comments! I am looking forward to see more.

[hr]

Ever since I published my first papers, I have been asked to participate in peer review. For me, peer review is an essential part of science. It is the part where (often anonymous) scientists will judge the work of other scientists and determine whether it is worthy of publication in the journal that the manuscript was sent to.

If you have ever submitted a manuscript for publication, you will know how nerve-wrecking and frustrating the peer-review process can be. Some reviewers can be very critical, denigrating, or even dismissive of your findings. They might demand additional experiments that will take months to perform, or they may just flat-out reject your paper.

The first time I was asked to review a paper, I was extremely honored (Finally! I am A Real Scientist!) but not sure what to do at all. Back then, there were not a lot of resources available online, and I did not have a lot of experience in critical readings of a paper. It definitely took many years of practice (“Eh, there is a typo in Table 2?”) to start to feel comfortable enough to suggest edits to other people’s manuscripts (“The findings described in this study are over-interpreted in the Discussion”). Even now, many years and about hundred peer reviews later, I am still not always sure if my reviews strike the right balance between being critical and fair. But I definitely try to do my best!

For the most part, peer reviewers do their work free and anonymously. It can be demotivating to never get credit for all the hours of work one puts in as a reviewer. But changes are underway.

My personal Publons profile page

In 2012, Andrew Preston and Daniel Johnston founded Publons, a new online community where scientists can get credit for their peer reviews. Richard van Noorden wrote a piece in Nature about Publons, called “The scientists who get credit for peer review“. And, if you want to see how the website works, here is a link to my online Publons profile. Recently, I was asked by Publons to be one of their advisors, and I gladly accepted. I think this is a great initiative and an innovative tool to make science more open and transparant.

Although I think that peer-review is an important and essential part of science, there are several flaws and frustrations that I regularly encounter, and I am planning to write about some of them here on a regular basis. So stay tuned!

[hr]