A new paper finds that peer review managers play an important role in identifying potential conflicts of interest (COI) in biomedical research grant peer reviews.
Peer review is the widely used process by which panels of experts evaluate research proposals to help funders identify the best research to fund. A cornerstone of the process is the integrity of the review panel, which includes a fair and non-conflicted evaluation of the proposed research. Despite this and the widespread belief that peer review is a foundation upon which U.S. science is built, few studies of the peer review process have actually been conducted.
And it can easily be abused. With the rise of tens of thousands of open access journals, and the most popular one being 'editorial review' rather than real peer review, peer review does not mean what it used to - but perhaps it never meant what the public thought. It may be only now that peer review lives up to its promise, though it will always be the case that if a credit card is available, some journal will publish a paper.
"Peer review is so central to the way we do science, it is important that we study the process. With good data, we can ensure the vibrancy of peer review and develop models and best practices that promote the integrity of peer review," said Dr. Stephen Gallo, AIBS Technical Operations Manager and the lead author of the new paper from the group, which has conducted peer review of research proposals on behalf of government and non-government research funders for over 50 years.
To inform the future of peer review, Gallo and colleagues conducted a retrospective analysis of COI data from peer review panels that evaluated 282 biomedical research applications. The overall 'conflicted-ness' of these panels was significantly lower than that reported for regulatory review panels, which have been studied by others.
The AIBS study found that 35 percent of conflicts were self-reported by review panel members. Importantly, peer review panel managers identified 65 percent of conflicts.
"The people who organize panels play an important role in identifying conflicts of interest," said Gallo.
Overall, this study suggests that the scientific community should dedicate some energy to improving COI reporting and detection methods. In light of increasing demands on reviewers' time, administrators will also need to make this process as efficient as possible while maintaining the highest ethical and review standards.
The study, Frequency and Type of Conflicts of Interest in the Peer Review of Basic Biomedical Research Funding Applications: Self-Reporting Versus Manual Detection, was published in the journal Science and Engineering Ethics.