The first time I was asked to peer-review an article, I was intimidated. How could I possibly cover all aspects of confirming if research was sound and ready to enter the academic literature?
To start, as with many situations, I called on friends for advice. While I’ve gone on a career path that mixes industry with teaching but doesn’t continue up the academic ladder, members of my various cohorts over the years are now snuggling into post-doc and even professorships, and reviewing papers is a basic part of their jobs. It doesn’t matter that they aren’t all in the same discipline as me — although the evolution of culture and landslides don’t share much overlap, reviewing papers in psychology and geoscience follow the same techniques applied to different fields.
The advice I was given is solid:
1. Why are you being asked to review the article? Are you a specialist in a particular model (for me, DAN-W and DAN3D), a technique, or a theory? Sometimes the editor will tell you directly, other times you need to guess. Evaluate that part particularly strongly, particularly checking if they applied the model correctly, did their math properly, or interpreted a theory consistently.
2. What is it for? A conference paper can reviewed more gently than an article for a prestigious journal.
3. By reading this paper, could you replicate the procedures or processes? Do you have confidence you understand exactly what they did and how they measured it? Is the process sound, or do they use an inappropriate combination (for example, using a container that is known to leach into the sample, or using the wrong model for the type of event)?
4. Would you cite this paper? Why not?
5. Copyediting is a very minor aspect of reviewing, but can be important when correcting the usage of technical vocabulary, especially for papers where the author is not writing in their native language. Be careful not to confuse style preferences with grammatical issues, but feel free to suggest alternate phrasings or correct typos.
Although I finished graduate school years ago, a good advisor-relationship can continue even after leaving academia. My advisor and I have a clear overlap in our academic interests, so any paper I am asked to review is one he will eventually read to stay current on the literature anyway. This means asking him to read it early and meeting with me to chat about it isn’t a big imposition on his time, so that’s exactly what I did. The biggest message he had for me was to relax and not stress about giving a flawless review. I wasn’t the only reviewer, and considering that peer review is debatably effective anyway, the entire academic body of landslide studies will not crumble if I make a mistake.
How do you go about peer reviewing articles?