Risk evaluation in peer review of grant applications

The process of peer review is used to identify the most scientifically meritorious research projects for funding. Impact and innovation are among the criteria used to determine overall merit. A criticism of peer review has been the perception that reviewers are biased against innovation, such as one study that found reviewers to systematically assign poorer scores to highly novel work. Moreover, reviewers’ definitions for excellent research and paradigm-shifting research are different; innovative research may not always be considered excellent. Therefore, it is clear more needs to be done to understand the decision-making processes of reviewers in evaluating risk and innovation in research. In an effort to address this gap, the American Institute of Biological Sciences developed a comprehensive peer review survey that examined, in part, the differences in applicant and reviewer perceptions of review outcomes. The survey was disseminated to 13,091 reviewers and applicants, of whom 9.4% responded. Only 24% of respondent applicants indicated that innovation was addressed in their review feedback, while 81% of respondent reviewers indicated they factored innovation into selecting the best science and 73% viewed innovation as an essential component of scientific excellence. Similarly, while only 27% of respondent applicants reported receiving comments on the riskiness of their grant applications, 58% of respondent reviewers indicated that the risks associated with innovative research impacted the scores they assigned to the grant applications. These results indicate a potential source of bias in how innovation and risk are evaluated in grant applications.