(This is the second post on a novel “parallel publication – peer review” journal. If your brain works in a linear fashion, check out the first post here. If you can live with “dropped threads” carry on – Jonathan (@sherbino) 🙂 )
By Brent Thoma (@Brent_Thoma)
In my post earlier this week I discussed my frustration with the traditional peer review process and pointed out the potential of publication platforms like The Winnower. With open-access peer review, free publication, social media for self-dissemination, and
altmetrics for measuring impact, non-traditional publishing platforms are becoming increasingly viable. Having recently published an article on the Winnower (Educational Scholarship in the Digital Age: A Scoping Review and Analysis of Scholarly Products [1]), I would like to share my actual experiences, both good and bad, with this new format of publishing.
The Benefits
Timely dissemination and publication
You cannot get timelier than instantaneous. The moment after I uploaded my formatted manuscript and clicked “submit” my manuscript was converted into professional HTML and PDF versions and made available online. We revised and updated the manuscript after the first two reviews and left comments responding to our reviewers. This process was repeated with another two reviewers before our manuscript was ready for publication. Approximately 1 month after our initial submission our article was officially published and granted a citable doi (digital object identifier).
Transparent and helpful reviews and responses
I found the review process to be extremely helpful and would like to thank our reviewers, Mahan Kulasegaram, Nishan Sharma (@NishanSharma29), Isabelle Colmers (@izzycolmers), and Allan McDougall (@AllanMcDougall). They provided respectful and insightful comments that helped to improve our manuscript. Their thorough analysis was best exemplified by Isabelle, who even found an error in one of our references! I believe that the open nature of the reviews (they are permanently available just below the HTML version of the article) assists readers in assessing the validity of the study and reassures them that a rigorous review has been conducted.
Beyond its open nature, the open peer review process took three other departures from traditional peer review
- As is often the case, we were given the opportunity to revise our manuscript and respond to our reviewers’ critiques. However, these responses were visible to both the article’s readers and our reviewers! Interestingly, this led to an academic dialogue where some of our reviewers commented on each other’s reviews to reinforce particular points.
- We were able to see each review as it was submitted and revise and resubmit our manuscript as frequently as desired (we resubmitted after our 2nd and 4th reviews).
- A star-based rating system was incorporated into the peer reviews that highlighted areas of strength and weakness for our manuscript. The mean score is displayed prominently at the top of the article. While I am not sure whether this will effectively identify standout articles, I was pleased to note that over the course of our revisions and the review process our paper jumped from a 4/5 to a 5/5 star ranking and hope that this indicated that our revisions resulted in noticeable improvement to the manuscript.
Open-access and free
The general rule in scholarly publishing is that you can have open-access OR free, but not both. While there are exceptions to this, the Winnower is one of few journals that truly provided both with no strings attached. As an added bonus, the article falls under a creative commons license, which allows it to be broadly disseminated on other platforms.
Altmetrics are tracked
Our article was not published in a high impact journal. However, the social media promotion around the article during its review process was incorporated into its post-publication Altmetric score [2] to reflect the impact of our work.
Amazing support
As outlined under “The Bad,” it was not all smooth sailing with The Winnower. I ran into several technical difficulties; however, the support provided by founder Josh Nicholson was far superior to all other publishers that I have worked with. I received same-day (often same-hour) responses to my questions and got the impression that he was truly trying to resolve them and improve his platform. While I doubt this level of service is sustainable, I suspect these efforts will have the platform running smoothly in the near future.
The Drawbacks
Recruiting reviewers is a challenge
This experiment helped me to appreciate the challenge that medical education journals face recruiting qualified reviewers. Our strategy for recruiting reviewers was to 1) request reviewers via our social media profiles and 2) identify individuals with expertise relevant to our topic (much in the way that we do when we suggest reviewers to journals) and contact them via email. Unfortunately, none of the reviews we received were made by people who we did not contact directly via email.
Prior to publication of our manuscript we contacted a total of seven potential reviewers via email. All of the reviewers responded and four agreeing to provide a review. We felt that these four reviewers had an appropriate knowledge base to provide commentary on all aspects of our manuscript. They included a physician who is an acknowledged expert on digital medical education (Nishan Sharma), a researcher with a masters degree who has published multiple systematic reviews (Isabelle Colmers), a researcher with a PhD in medical education (Mahan Kulasegaram), and a PhD-candidate in medical education (Allan McDougall).
Having now published with the Winnower, I would be more inclined to provide a review for others publishing on the platform. Ideally, as the journal grows it will develop a community of individuals that are readily willing to take on this role.
Authors with a smaller social media profiles may have less readers
My co-authors and I have a significant social media footprint, which has made disseminating this work possible. In particular, we are all active on Twitter (all four authors have personal accounts with 1900 to 8500 followers) and have blogs that can be used to promote our publication. While the Winnower also has social media accounts that it uses to promote publications, it is unlikely that authors who have not built a presence on social media will be able to disseminate their work as broadly.
The article will not be indexed in notable repositories
As the Winnower is not (and is unlikely to be for many years) an indexed journal, its availability in traditional repositories is limited. That said, the article was immediately discoverable on Google Scholar after coming online. (A Google Scholar search strategy is increasingly being recognized as a viable/essential component of a literature search strategy [3,4].)
Conclusion
Publishing in a non-traditional journal was a mixed bag. As the current academic climate gives less credit to publications in alternative journals, I will not be committing a substantial amount of my work to this format in the future. However, I do think it is a viable platform for the dissemination of selected scholarship. I will never again let an unpublished (but publishable) manuscript get stale waiting for a sympathetic reviewer. I would encourage other medical researchers to give the Winnower a chance so that it can build the credibility and community that it needs to help make open-access, open review publication a credible option.
Acknowledgements
I would like to thank Teresa Chan (@TChanMD) and Michelle Lin (@M_Lin) for providing a review of this post.
Conflict of Interest Declaration
I have no conflicts of interest to disclose in regard to the topics discussed in this blog post.
References
- Thoma B, Chan T, Benitez J & Lin M. (2014). Educational Scholarship in the Digital Age: A Scoping Review and Analysis of Scholarly Products. The Winnower. doi:10.15200/winn.141827.77297.
- Brigham, T. J. (2014). An Introduction to Altmetrics. Medical reference services quarterly, 33(4), 438-447. doi:10.1080/02763869.2014.957093
- Gehanno, Jean-François, Laetitia Rollin, and Stefan Darmoni. (2013). Is the coverage of Google Scholar enough to be used alone for systematic reviews. BMC medical informatics and decision making, 13(1), 7. doi:10.1186/1472-6947-13-7
- Bramer, W. M., Giustini, D., Kramer, B. M., & Anderson, P. F. (2013). The comparative recall of Google Scholar versus PubMed in identical searches for biomedical systematic reviews: a review of searches used in systematic reviews.Systematic reviews, 2(1), 1-9. doi: 10.1186/2046-4053-2-115.
Image 1 courtesy of The Winnower
Image 2 courtesy of Creative Commons Canada
Image 3 courtesy of Google Scholar
