By Jonathan Sherbino (@sherbino)
“Crowdsourcing is based on the framework of collective intelligence, the idea that knowledge is the most accurate when it consists of inputs from a distributed population –“all of us together are smarter than any one of us individually”.
(from Aitamurto T, A Leiponen, R Tee. (June 2011) The Promise of Idea Crowdsourcing – Benefits, Contexts, Limitations [white paper]. Retrieved from: http://www.crowdsourcing.org/document/the-promise-of-idea-crowdsourcing–benefits-contexts-limitations/5218)
Crowdsourcing is trendy. Kickstarter uses crowdsourced funding for business projects/art installations/whatever that cannot get traditional financing. Big business uses crowdsourcing to produce low-budget internet-based, viral advertising, yielding a big return on investment. But as with every trendy ideas, aspects of it have run its course. According to researchers at Stanford, “Idea crowdsourcing has become a hype term associated with unrealistic expectations for innovation and unclear understanding of its requirements and challenges.”Now that the ‘cool factor’ has worn off crowdsourcing, there is room to evaluate its long term, sustainable merits.
Traditionally, in academic publication, work is sent to three “peers” who are blinded to your identity, and you are usually blinded to theirs. These three people determine if your manuscript is accurate and, if they approve, the manuscript gets published. In this age of social media, would it not be more efficient to adopt post-publication review, where multiple individuals can tack comments/disagreements to your publication and therefore validate the criteria of ‘peer review’ for publication? (See here for previous comments)
The downsides of crowdsourcing for peer review include (but are certainly not limited to…!):
- lack of ‘expert’ (i.e. appropriately knowledgeable) opinion,
- regression to the mean (i.e. multiple measures/opinions collate around a average which often has limited discrimination)
- potential manipulation of the scoring system (i.e. bias – If I can buy 1000 Twitter followers for pennies, what’s to stop big pharma etc. from buying crowdsourced opinions to validate their claims
But what about the upside of crowdsourcing? Post your ideas below. Let’s try a “meta”. The upside of crowdsourcing from the crowd is…
Image courtesy of ddpavumba / FreeDigitalPhotos.net
