Why ex post peer review encourages high risk research while ex ante review discourages it


Importance

Science operates within social structures that govern and shape scientific activity. One such institution is peer review, which focuses attention on promising and interesting science while encouraging scientists to look at some issues rather than others. Here we show that the ex ante review of proposals for future work and the ex post review of completed scientific work create different incentives for researchers. This tension creates a dilemma, as most researchers must find projects that will survive ex ante and ex post peer review. By breaking down this dynamic, we can understand how peer review shapes scientific activity and how changes in peer review can take science in unforeseen directions.

Summary

Peer review is an integral part of contemporary science. While peer review focuses attention on promising and interesting science, it also encourages scientists to look at some issues at the expense of others. Here, we use insights from the prediction assessment to examine how two modes of peer review – the ex ante review of proposals for future work and the ex post review of completed science – motivate scientists to prioritize certain issues over others. Our main result is that ex ante and ex post peer review pushes researchers to distinct sets of scientific questions. This tension arises because the ex post review allows researchers to leverage their own scientific beliefs to generate results that others will find surprising, while the ex ante review does not. In addition, the ex ante review will promote different research questions depending on whether reviewers rank proposals in anticipation of changes in their own personal beliefs or those of their peers. The tension between ex ante and ex post review puts investigators in a bind, as most researchers must find projects that will outlast both. By breaking down the tension between these two modes of review, we can understand how they shape the landscape of science and how changes to peer review might shift scientific activity in unforeseen directions.

Footnotes

    • Accepted 23 October 2021.
  • Author Contributions: KG and CTB designed research, performed research, contributed new analysis tools, and wrote the article.

  • The authors declare no competing interests.

  • This article is a direct PNAS submission.


Comments are closed.