At the Student Network for Open Science (NOS), we aim to rethink scientific publishing – both by giving students the possibility to publish their research, and by making our journal transparent and accessible to students and early career researchers. While students learn how to navigate the academic world better by stepping into the role of a reviewer or member of the editorial team at NOS, we are also discussing current publication practices critically. Our peer review fellowship offers deeper insights into current practices of peer review and ongoing discussions about how to reshape scientific publishing. During the course of six sessions, we invited several experts on different aspects around peer review, covering the topics of criticism on peer review, possible alternatives, bias, open science, and the perspective of a former editor of an established journal. In this article, we want to reflect on the fellowship, share our learnings, and spark your interest in future participation.
What is Peer Review – and how good is it, really?
Peer review is a common practice in scientific publishing today, in which a piece of research is evaluated by independent researchers upon submission to a scientific journal. This review usually suggests changes or additional analyses, and ultimately gives a recommendation whether to accept or deny publication. Therefore, peer review is often viewed as a quality control mechanism, as experts from the field are expected to assess the quality of an article. However, there is a growing number of anecdotes in which peer review failed to flag inaccuracies, leading to the dissemination of false information. Another aspect often criticized about peer review is its lengthy process that impedes fast and effective knowledge sharing, especially in situations when international scientific collaboration is crucial, such as the COVID-19 pandemic. Peer review in its current form is criticized for many other reasons, including the fact that reviewers do not get paid for their contribution. Potential conflicts of interest arise, as both original author and reviewer publish in the same field, thus competing for funding and attention. Anecdotal reference from the scientific community, as well as some reviews, have shown that reviews often do not succeed in adding value to a scientific paper, and can even contain unprofessional comments that are inappropriate in the assessment of a scientific article. These inappropriate comments, they reflect a larger problem: Bias occurs on many levels in peer review, e.g., the name and institution of an author can tip a reviewer to be more in favor of publication, hence asking for lighter revisions in their review. In the fellowship, we discussed the many aspects of bias and recognized how imperative it is to critically discuss predisposing factors in peer review.
What if…? Reimagining Peer Review
The extensive criticism on peer review raises the question of how peer review might look like in the future, and whether there are any alternative frameworks. The Open Science movement takes a prominent role in the criticism of today’s publishing practices. As research questions grow more complex than ever and make collaboration imperative, the internet provides to chance to build a new network of open data, methodology and knowledge exchange. Open Science advocates for open access, methodology, source, educational resources and peer review, trying to build a more collaborative framework of knowledge dissemination. Key features of open peer review include transparency about the publication process, i. e. reviews are to be published, allowing to retrace the steps that led to final publication of a research piece. Closely related to Open Science, we dedicated a fellowship session to possible future approaches to peer review. An initiative currently gaining momentum is Publish Your Reviews, that calls for reviewers to upload their submitted reviews on an online platform. This gives readers of the original preprint or finished manuscript the opportunity to have deeper insights into the evaluation process; additionally, it rewards the reviewers with an openly accessible and citable scientific contribution, and invites the broader community to an ongoing discussion about both the article and the review.
In conversation with a former editor at Nature Communications, we reflected on how these new approaches can be introduced to the current publishing formats. With growing awareness of the imperfections of peer review, combined with dissatisfaction about academic publishing in general, the scientific community has begun a transformative process towards more open publishing structures: Preregistration opportunities, extended methodology and code sharing, as well as open access availability of articles are three of the earliest achievements of the movement calling for more transparent publishing practices.
Join us in discussing the future of scientific publishing!
In six sessions, we discussed peer review from many different angles and got an elaborate image of current problems and future opportunities. The peer review fellowship opened a valuable discussion about how we as future scientists want to approach peer review – both as scientists and contributors to the NOS journal. We are thankful for the meaningful input by our guest speakers Finn Lübber MSc from Lübeck University, Prof. Dr. Ludo Waltman from Leiden University, Prof. Dr. Lisa Roberts and Prof. Dr. Shannon Peters from Boston University, Dr. Lena Karvovskaya and Dr. Stephanie van de Sandt from Vrije Unversiteit Amsterdam, and Dr. Christine Mieck from Charité Berlin. If you are interested in joining the next NOS peer review fellowship, sign up for our newsletter to stay informed about the next application deadlines.