Introduction
[Log in, then click "edit" top right to start writing / to see the Authorea toolbar and other features]
[Elizabeth and all: Please use "Insert / Citation" to add references throughout, then they will auto complete as shown below] .
Introduction
'Transparency in review' was the theme of Peer Review Week, 2017 \citep{meadows}. Eliz to add Something about the surveys in 2015 (favouring double-blind.). However, recent surveys across researcher communities have shown growing support for more openness in peer review perhaps encouraged by growing support for open research initiatives more generally \citep{reviewers}, \citep{2019} [any others?]. Publons’ Global State of Peer Review also uncovered important trends regarding peer review practices including a greater willingness among younger researchers to adopt transparency in peer review \citep{review}. Journals are increasingly recognising the benefits of introducing transparency. A transparent peer review workflow shows readers the process behind editorial decision-making, increases accountability, and helps recognize the work of editors and peer reviewers \citep{godlee2002}. EMBO press, one of Wiley’s publishing partners, introduced transparency into their processes in 2009 and their experience has been overwhelmingly positive \citep{pulverer2010}.
In August 2018, two biomedical funders (Wellcome and Howard Hughes Medical Institute) and ASAPbio (a non-profit organisation encouraging innovation in publishing) advocated publishing peer review reports \citep{polka2018}. Their open letter in support of the benefits of peer review transparency has over 370 journal signatories and is still growing \citep{letter2018}. There are also guidelines for those wishing to implement transparent peer review workflows \citep{gorogh2019} and recommendations for publishing peer review materials \citep{al2018}. However, despite the growing interest in transparent review, publishers have been hindered in their ability to adopt new approaches given limitations of existing workflows.
In September 2018, Wiley announced the first cross-industry initiative to pilot an automated, scalable transparent peer review workflow, in collaboration with Publons and ScholarOne (both part of Clarivate Analytics) \citep{clarivate}. The purpose was to enable journals to 'open up' peer review by introducing more transparency into the process. A vital part of this was the flexibility to enable authors to have the option of transparent peer review when they submit to a journal that can still otherwise offer more traditional models of peer review if authors decline transparency. By transparent peer review in this context, we mean that authors are "opted-in" for transparent peer review unless they opt-out. If authors opt for transparency and their article is published, then the peer reviewers’ reports, authors’ responses, and editors’ decisions will accompany the published article. Reviewers also have the option to disclose their names alongside their reports but this is not mandatory. The peer review history is freely available via a page hosted by Publons linked from the published article, for example, \citep{chen} and this also allows engagement with the peer review process to be measured. Each component of the peer review history has a DOI, ensuring each element is fully citable. For those reviewers who choose to sign their reviews, the DOIs can also be added to their ORCID records. The layout of the peer review page reflects the sequence of peer review from original submission, initial peer review, revision, re-review to acceptance of the version of record. Furthermore, each peer review element can be endorsed by, or commented on, by Publons users.
The first journal to join the pilot was Clinical Genetics \cite{graf2018}. Since then, the initiative has grown steadily at Wiley, including a further 10 journals in January 2019 \citep{moylan2019} and reaching a total of 40 journals for Peer Review Week, September 2019 \cite{graf2019}. This September 2020, the theme of Peer Review Week is 'Trust' \cite{meadows2020}. Transparent peer review is, of course, a significant step towards bringing greater trust to research publishing and recognition to all involved in preparing and disseminating research. Making the peer review history visible and discoverable is key to improving the quality and efficiency of research communication and improving researchers' trust in a journal's processes while also potentially reducing fraud in peer review.
This September 2020, 65 {check} Wiley journals are now offering transparent peer review across a range of disciplines. Here we reflect on what we have learnt during the past two years and share our findings. We present data on the transparent peer review pilot from 27 journals that have participated for a sufficient length of time to enable robust data collection. In order to minimize bias we assessed similar data on a set of 29 comparable 'control' journals that did not introduce transparent peer review across the same time period. We wanted to understand the effect of introducing transparent peer review to a journal in terms of turnaround times and willingness of reviewers to agree to review.
Methods
Data from 27 journals that have been participating in the transparent peer review pilot were compared to data from 29 'control' journals that did not introduce transparent peer review but were sufficiently similar in terms of discipline, impact factor, number of submissions and number of publications that no significant differences were detected by using t-tests and Chi2 tests.
We evaluated data from the journals that joined the pilot pre- and post- the introduction of transparent peer review and compared this to data from the control journals for the same time period (1st September 2018 to 31st March 2020). The majority of journals implementing transparent peer review began doing so in September 2019 . Articles in the control group were randomly assigned "pre/post TPR" status based on the month of their publication and the distribution of "pre/post TPR" status in the test group during that month. For inclusion in the study, submissions from both groups of journals must have reached a final decision.
All results are presented with xxx (t tests?) to show significance. {Conni I think we need a simple line that explains this?) EM: Do we need to add to figures then?
Results
//
[Conni: Please explore how tables and figures work ("Insert / Figure" and "Insert / Interactive figure" and other options under "Insert"), and also talk with Dave Flanagan about how to use plot.ly to make possibly interactive figures like we did here \cite{Tan_2020} -- also please ask about how to upload processed data file alongside figures and tables, like appears next to Fig 1 and Fig 3 at that same reference]
Conni, I have 'simply' included tables and figures as shown below. Tables are inserted from excel. Figures I saved as individual powerpoint slides, then saved as a PDF and inserted as a PDF. I am not sure this is 'optimal' but I think it could suffice, but Dave may have more tips. thanks Elizabeth
AS A REMINDER, THE DATA INCLUDED IN THIS PREPRINT USE
The recording of Conni's insights session here (starts ~15 mins from end):
And also data in excel here: