Evaluation is not just a matter of threading a judgement through a procedure, but also of characterizing the quality of something. — Stefan Hirschauer: Editorial Judgements
Double-blind peer review is one of the pillars of the scientific apparatus. Many manuscripts are evaluated by their peers before they are being published in order to ensure quality standards and constantly redefine the scope of a field.
I mostly enjoy reviewing — no matter the quality of the paper –, since I can contribute to an idea and improve a manuscript. However, sometimes, there is this sneaky paper, where you can see that someone has been left alone by their peers. Those manuscripts have obviously not received any love or critical feedback and are rough 2nd or 3rd drafts of writing. I love discussing the content, but if I’m overwhelmed by the amount of grammar, style and structure issues I need to mention I feel like someone used the review process for proof reading and editing feedback more than actually expecting to get published. Doing this type of work for friends and colleagues is ok and I offer it willingly, but having the job dumped on me feels weird. Now, I could always reject reviewing a paper, but then I feel like I’m just deflecting the job to some other poor soul who has to go through the motions regardless. One solution there would be that publication venues offer mentors who are willing to proof read papers of more junior writers and give them structural feedback. Most academic writing is published in English, which privileges native speakers. For grammar and style issues — mostly from non-native speakers –, we should think about having an optional editing stage before or after reviewing — and communicating that to reviewers. These things cost money and time, yes, but then publishers make tons of money from work they don’t pay for anyway, so why not invest into that, so that we all can enjoy reviewing papers more and getting enthusiastic about ideas.
There is another side to reviews as well: the receiving side. I mostly enjoy receiving reviews — no matter the result –, since they always help to improve an idea as well as the manuscript and, ultimately, that’s what I want: the best version of a manuscript possible. Some reviews are unhelpful, yes, some reviews didn’t quite understand what you were writing about, yes, but in the end it tells you, that you have to be more concise in your writing. However, sometimes we receive reviews that are unkind and unappreciative towards the work you describe in your manuscript. Sometimes we receive reviews that tell us we chose not only the wrong venue (fair enough), but also that we have no place in the chosen field. This seems awfully ignorant of the fact that almost no publications come from authors who are not embedded in public or private research institutions in which they have to work with others who are also part of their field. The anonymity of authors and reviewers alike during the review process is essential to counteract the most obvious favouritism. However, often authors are already obvious to reviewers during the process or become obvious when a manuscript is published. Reviewers remain anonymous other than when they choose to identify themselves to the authors after publication and even that is frowned upon. I argue that anonymous reviewers behave somewhat like anonymous commenters on the internet: they are ultimately harsher. Reviewers are not accountable for what they write and, hence, sometimes don’t filter their criticism with the appropriate appreciation for the work in front of them. Making it standard that reviewers are disclosed after the review process is finished and a decision has been made, encourages them to stay more focused on their evaluation task and creates a more helpful and supportive review system that we all can profit more from.