The future of peer review

The European Science Foundation held a conference about peer review last year, and debated how to make the process better. This was not something I was aware of, until I saw this ScienceDaily article - Quality On Peer Review Must Be Raised With Co-operation, Says Report

Scientists are questioning whether peer review, the internationally accepted form of scientific critique, is able to meet the challenges posed by the rapid changes in the research landscape. A report published by the European Science Foundation (ESF) has showcased a number of options that could lead to a greater openness to innovative research.

The report “Peer review: its present and future states”, which draws on ideas from an international conference in Prague in October 2006, reflects some concern on the shortcomings of peer review while outlines some possible measures to cope with them.

[...]

A central theme of the report is that the current peer review system might not adequately assess the most pioneering research proposals, as they may be viewed as too risky. John O’Reilly, former Chief Executive of the U.K.’s engineering and Physical Sciences Research Council (EPSRC), now Vice Chancellor of Cranfield University, said traditional peer review might be too risk averse. He suggested the need to encourage pioneering research that is high risk in the proposal, but high impact if successful.

The conference called for new approaches, enabling the assessment of innovative research, to be embedded in the peer review system. An example of a new approach to overcome the perceived risk-averse funding culture was given by Dr. He Minghong from the National Natural Science Foundation China. His Foundation encourages reviewers and programme managers to spot risky project proposals which are then funded under stricter conditions. Their duration is shorter, their budget smaller and they are more closely monitored.

I am on two mind on this. There is no doubt that they are right that pioneering research has a hard time in traditional peer reviewed journals, but I can't help worry about the number of bad papers that will be published if there is a less critical review of such papers than there is now. The Chinese approach does sound good, but it can only be applied to studies funded by public grants.

Hopefully the focus on the issue can bring new ideas to the table, which will allow the peer review system to continue, while allowing more room for pioneering works. Peer reviews are critical for proper science.

ScienceDaily unfurtunately don't link to the papers (or even press releases) that they are covering, so I haven't been able to locate the report, “Peer review: its present and future states”. The conference in Prague have a website though, which has the presentations from the conference.

Edited to add: I found both the press release, and a link to the report (.pdf).