Study Links Tough Peer Reviews to Higher Scientific Impact

An analysis of publicly available peer reviews for thousands of papers found that papers receiving tough reviews went on to have a higher impact in science than those that sailed through the review process. The study, posted to the arXiv preprint server last month, evaluated the peer-review correspondence associated with a selection of papers published in Nature Communications.

The journal has been making these files public for papers that the journal accepts since 2016, as long as the papers' authors give their consent.

The journal says it releases the files for transparency and to inform discussion of published papers in the research community. Co-author An Zeng, a specialist in complexity science at Beijing Normal University in China, says that he and his colleagues thought these files could tell researchers a lot about the negotiation between reviewers and authors to get papers published.

To draw insights from the review correspondence, the team prompted a large language model to evaluate these files for 8,000 published papers. The researchers selected 1,000 papers randomly per year from the period 2017 to 2024. Scientists are not usually thrilled when reviewers of their research papers ask for extensive revisions ahead of publication, but the stress that authors experience might pay off in the long run.