Sunday, November 13, 2011

Peer review and the corruption of science

Peer review is the process that decides whether your work gets published in an academic journal. It doesn't work very well any more, mainly as a result of the enormous number of papers that are being published (an estimated 1.3 million papers in 23,750 journals in 2006). There simply aren't enough competent people to do the job. The overwhelming effect of the huge (and unpaid) effort that is put into reviewing papers is to maintain a status hierarchy of journals. Any paper, however bad, can now get published in a journal that claims to be peer-reviewed.
The blame for this sad situation lies with the people who have imposed a publish-or-perish culture, namely research funders and senior people in universities. To have "written" 800 papers is regarded as something to boast about rather than being rather shameful. University PR departments encourage exaggerated claims, and hard-pressed authors go along with them.
Not long ago, Imperial College's medicine department were told that their "productivity" target for publications was to "publish three papers per annum including one in a prestigious journal with an impact factor of at least five.″ The effect of instructions like that is to reduce the quality of science and to demoralise the victims of this sort of mismanagement.
The only people who benefit from the intense pressure to publish are those in the publishing industry. Hardly a day passes without a new journal starting. My email inbox is full of invitations to publish in a weird variety of journals. They'll take just about anything. The US National Library of Medicine indexes 39 journals that deal with alternative medicine. They are all "peer-reviewed", but rarely publish anything worth reading. The peer review for a journal on homeopathy is, presumably, done largely by other believers in magic. If that were not the case, these journals would soon vanish.
But it isn't only quack journals that have failures in peer review. In June, the British Journal of General Practice published a paper, "Acupuncture for 'frequent attenders' with medically unexplained symptoms: a randomised controlled trial (CACTUS study)". It has lots of numbers, but the result is very easy to see. All you have to do is look at their Figure.
http://www.guardian.co.uk...rish-peer-review-science