We delved deeper into this topic of preprint journal clubs being a source of peer review training. We wondered how many of our respondents had ever contributed to the peer review process (72%), and whether they had ever received training (18%). These results were alarming and, not surprisingly, 86% felt writing a preprint review would provide beneficial peer review training. Encouragingly, 78% of respondents said they would be willing to write a preprint review, but the majority (60%) would only be willing to spend an hour writing it. Several of our respondents suggested taking a different approach for preprint journal clubs to reduce the time commitment and burden of writing a preprint review: to have a two-person team for each preprint journal club, allowing one to take notes while the other presents the slides. In addition, we suggest a collaborative approach where all journal club attendees provide quick feedback by answering the questions listed in our
PREreview quick participant worksheet . The responses can be collected by the journal club host and incorporated into the review.
While most of our participants were in favor of preprint reviews, as a peer review training exercise for early-career researchers, what about their views from an author’s perspective? Would they be supportive of feedback from the scientific community when constructing/finalizing their manuscript and, if so, would they prefer that feedback to be open (posted online), or private (emailed to them)? First, of the 94% that had written or contributed towards writing a scientific manuscript during their career, a massive 73% would have found it useful to have feedback early on. But, when it came to how this feedback should reach them, opinions were much more divided, with equal preference for open versus private (37% and 36%, respectively) and 24% showing no preference. Of note, 2% of our respondents brought up the issue of whether an openly-posted review should be signed, i.e. the reviewer’s identity revealed. This is a difficult topic as many feel that anonymous commenting can encourage non-constructive feedback and possible harassment. On the other hand, some early-career researcher may feel uneasy about having their identity known due to potential negative repercussions on their career. It is likely these opinions will remain divided unless open peer review becomes the norm.
Conclusions and future directions
When we launched this survey we had no idea the community would be so overwhelmingly positive and supportive about the integration of preprints into journal clubs. We were energized by the results, and this motivated us to push forward with our PREreview platform for collaborative preprint review. Although we acknowledge that the social media networks we targeted were likely to attract open science enthusiasts, the sampled community was actually quite divided over whether preprint reviews should be openly posted or private. These results mirror the community's response to
similar questions about open editorial peer-review practices, and is therefore a subject requiring care and consideration. When deciding whether to build PREreview, we recognized the huge benefit of generating a platform populated by many preprint reviews that would serve as a training resource for early-career scientists wanting to learn the art of peer review.
Our vision at PREreview is to deeply integrate preprints into a streamlined science communication system, where manuscripts are written collaboratively, posted openly on a preprint server and passed through open peer review for community assessment and evaluation. We know this vision is still a few steps from being the reality, but we can only see where the steps will lead if we take the initiative to explore.