Automation of literature reviews
A recently published paper in Scientometrics raises the specter of an automated tool that would search through existing citations and “facilitate novices to perform tasks that are usually carried out by trained professionals.” The tool was then used for students to create literature reviews and these were submitted to conferences. The tool was so successful, claimed the author, that
“Among the seventeen papers submitted to conferences, fourteen papers were accepted for oral presentations, and one was accepted as a poster. The acceptance rate is 15/17 = 88.2%. One of the papers submitted to the 2011 industry innovation and technology management conference … was awarded the best paper in the conference.”
“Graduate students utilizing this system are able to have their papers published in their first semester in school.”
So, what do you think? Should this form of citation analysis and automation be allowed?
Chen, T. T., 2012: The development and empirical study of a literature review aiding system. Scientometrics, 92, 105-116.
(Image from http://ukliteraturereviewexample.blogspot.co.uk)
If this is only used as a tool to help with the review process then I have no problem with it. However, if it is used blindly then I foresee problems.
I think the real question is whether the software is sufficiently well-designed that it produces a lit review that’s indistinguishable from a “real” one in a peer-reviewed manuscript. If so, then sure, it could be allowed. However, I suspect this is unlikely to be achievable for all fields, since the styles and standards of lit reviews in different fields (and different journals) are not uniform.