Discussion
When the transparent peer review initiative began at Wiley, we faced some crucial decisions about how best to encourage the cultural change necessary to facilitate more openness in peer review. We took the decision to make transparent peer review optional for authors, but to implement this as an 'opt-out', i.e., authors were opted-in to transparent peer review unless they opted-out. This approach is in keeping with Wiley's support for open research practices \cite{x9ods} and may explain the high numbers of authors who remain opted-in \citep{gorogh2019}. On average, 86% of authors remained opted-in to transparent peer review across the 27 journals offering transparency. This compares with findings from a trial at Nature Communications where 60% of authors voluntarily opt-in \citep*{editorial2016} and findings from PLOS of 39% uptake when transparency is offered as an opt-in \citep*{kennedy2020}. However, differences in uptake may also reflect differing support across research communities and subject disciplines \citep*{harris2020}.
When offering transparency as an 'opt-out', there also has to be flexibility if authors changed their minds. Overall, a small proportion of authors (244) changed their mind about their preference of peer review model when submitting a revised version, with 57% wanting to adopt transparent peer review and 43% wanting to opt-out. Journal workflows need to be configurable to accommodate situations where preferences change.
When reviewers are invited to review transparently, they have to agree to sharing their report content if the authors article is published, but it is optional whether they sign their reviews or not. Most reviewers chose to remain anonymous, with only 15% of reviewers willing to sign their reviews. This suggests that peer reviewers are less comfortable with sharing their identity; other studies have also reported a preference among reviewers for anonymity \citep{bravo2019}, \citep*{harris2020}.
Comparing data from the journals pre- and post- the introduction of transparent peer review with control journals for the same time period enabled us to determine if any of the effects we saw could be attributed solely to transparent peer review or if there were other factors involved. Offering transparent peer review did not seem to have an adverse effect on submissions. In the time period of this study, submissions increased across both the journals offering transparent peer review and the control journals. Similarly, there was no adverse effect on times taken to reach an initial decision, final decision or number of revisions an author had to submit under the transparent peer review model. Overall, both the journals operating transparent peer review and the control journals were faster post the time period when transparent peer review was offered and may reflect other general improvements in editorial processes.
Interestingly, in cases where a journal offers transparent peer review, we did observe an increase in the number of decisions that are made directly by editors without external peer review. We also noted that editors made more referrals to other journals than they did prior to operating under transparent peer review. Interestingly this effect was larger in the transparent peer review group than the control group, even though more journals in the control group were participating in journal referral networks than the transparent peer review group.
Reviewers are slightly less likely to complete their report for the journals that began operating transparent peer review (Figure 6) but this effect was not seen in the control journals. In order to ensure a manuscript receives a sufficient number of agreed reviewers, editors often have to invite more peer reviewers when the journal operates transparent peer review (Figure 7). However, other studies of transparent peer review have not found this to be the case \citep*{cosgrove2018}, \citep{bravo2019} suggesting that this may vary across subject disciplines. Certainly studies of fully open peer review where reviewers make visible both their report content and their identity indicate that more reviewers have to be invited \citep{rooyen1999}, \citep*{samarasinghe2017}, \citep*{burley2017}. However, the continued success of journals offering more open models of peer review suggests that editors are able to overcome any perceived challenges in securing sufficient reviewers for the added benefits that increased transparency brings \citep*{amsen2014}.
As the transparent peer review initiative has grown at Wiley, we have been encouraged by feedback from editors \citep*{graf2018}, \citep{moylan2019} and authors who have valued the fair and high-quality peer review comments they have received. Our approach to offering transparent peer review is fully compatible with journals that operate double-blind peer review and single-blind peer review. So a journal could be configured to offer double-blind peer review to address any perceived bias during the peer review process, but still operate transparently post-peer review in sharing review content. The flexible way in which we offer transparent peer review also allows authors and peer reviewers to choose a level of openness they are comfortable with when undertaking their review.
Given the success of the pilot initiative to introduce more transparency to peer review and the lack of any overwhelming detrimental effects, we will continue to offer transparent peer review to more journals in future.
Limitations
The study is limited by the number of journals that were taking part at the time the study was undertaken (27 journals). Journals joined the initiative at different times, so for some journals we only had six months of data, others more. Journals included in the pilot were also 'self selecting' in their willingness to embrace transparent peer review. Any differences between authors and reviewers in terms of subject areas, countries and gender, for example, have not been explored. In addition, different journals have different policies with how reviewers are invited, either sequentially or in larger numbers at a time, and this may have a bearing on comparisons between journal groups.
Conclusion
Two years on from starting to offer more transparency in peer review via a collaborative pilot initiative we are delighted that author uptake remains high. Overall, we are not seeing any adverse effects on editorial turnaround times for journals that have introduced transparency. However, we acknowledge the additional editorial effort involved for editors who have to invite more reviewers in order to secure a sufficient number to peer review an article. This additional effort, although significant, is not insurmountable and brings the added benefits of trust and transparency to the peer review process, accountability and recognition for the work done.
Data availability statement
In order to protect the identity of journals taking part in the transparent peer review initiative we have only included summary data in this preprint. Tests and data that support the analyses are shared in the Appendix.
Disclosure of conflicts of interest
All authors are employed by Wiley and benefit from the company's success.
Author contributions
Elizabeth Moylan: conceptualization, writing - original draft, writing - review & editing. Kornelia Junge: data curation, formal analysis, investigation, methodology, visualization, writing - review & editing. Candace Oman: visualization, writing - review & editing. Elisha Morris: conceptualization; investigation; writing - review & editing. Chris Graf: conceptualization; supervision; writing - original draft; writing - review & editing.
Acknowledgements
Thank you to the dedicated teams at Wiley, Publons and ScholarOne that enabled this initiative to happen and continue to grow. We are also extremely grateful for the helpful feedback and comments on our initial findings from Peter Creaton, Gareth Jenkins, Elizabeth Matson, Helen Pedersen and Michael Willis.
Appendix
The tests and data that support the analysis and some additional data are available here: