AI Feedback Prompted More Manuscript Revisions in a 31,020-Paper Trial

A manuscript can be technically ready for the world and still be starved of something ordinary: another person’s careful attention.

Feedback is a quiet form of scientific infrastructure. It catches unclear arguments, exposes weak assumptions, and helps a researcher decide what to fix before formal review. But it is not evenly available. Senior researchers at well-connected institutions can draw on deep networks of colleagues. Early-career scientists and teams farther from prominent academic circles may have fewer people to ask—and longer waits for useful critique.

A large field experiment asks whether artificial intelligence can widen that bottleneck without taking the researcher’s judgment away.

A live test across 31,020 manuscripts

In early 2024, researchers from Northwestern, Stanford, Emory, and Cornell collected 34,340 manuscripts newly posted to arXiv and randomized them by paper across 150 fields. The main analysis excluded 3,320 papers that had already been revised before feedback could arrive, leaving 31,020 manuscripts and 45,466 contacted authors across 133 geographic regions.

Authors assigned to the treatment group received an email linking to a private page with feedback tailored to their manuscript. A fixed, multi-agent language-model pipeline produced review analysis, alternative title suggestions, and grammar checks. The control group received no message.

During the month after delivery, treatment papers averaged 0.005 more revision events per manuscript than control papers. That is roughly one additional revision for every 200 manuscripts assigned to receive feedback, and a 12.55 percent increase relative to the low control baseline. The estimate was statistically significant at the study’s reported threshold.

Both scales matter. “Twelve percent more” sounds dramatic without the absolute number beside it. The experiment did not make every paper produce a new version. It showed that a single, light-touch intervention changed revision activity by a small amount across a very large population.

Research references: [1]

A signal about access—not proof of equalization

The authors report statistically significant treatment estimates among manuscripts linked only to non-English-dominant environments, papers with fewer references, lower-h-index teams, and earlier-career authors. Within those lower-category groups, the reported relative increases in revision activity ranged from 18.9 to 26.4 percent. Corresponding estimates for the higher-category groups were small and not statistically distinguishable from zero.

That pattern is consistent with a practical possibility: AI feedback may be especially useful where human feedback is scarce. But the published analysis does not establish that treatment effects differed statistically between the groups. It is not proof that the tool reduced inequity, and it does not establish why each author revised.

The useful design question is still worth pursuing: can AI serve as an additional source of critique for people who have fewer routes to it—without replacing collaborators, mentors, or reviewers?

Research references: [1]

What the experiment did not measure

The observable outcome was the number of revised arXiv versions posted within a month, not whether independent experts judged the science better. An exploratory text analysis found little difference in surface edits and only directional evidence of more substantive changes. The experiment therefore does not show improved validity, novelty, reproducibility, or research quality.

The no-contact control also cannot separate the value of the feedback itself from the attention generated by receiving an email. And a later finding—5.29 percent higher measured language-model adoption intensity among authors with minimal prior detected use—depends on an AI-text detector. The authors note that such detectors are noisy and may behave differently for non-native English writing.

Northwestern University’s institutional review board approved the study under protocol STU00220102. The open record remains arXiv version 1, submitted May 22, 2026. An Academy of Management Proceedings abstract is also indexed under DOI 10.5465/AMPROC.2026.21478abstract, but this analysis is based on the open arXiv manuscript rather than a later full journal article.

Research references: [1] [2] [3]

More feedback, with human judgment intact

The experiment advances the conversation beyond demonstrations of what a model can say. It asks whether people act when feedback reaches them, and it measures that behavior in live scientific work.

The human-agency question is not whether an AI can “be the scientist.” It is whether a researcher gains another useful chance to reconsider a sentence, a claim, an ethical issue, or an argument—and remains free to reject the suggestion. A good system would make critique more available while keeping authors accountable for evidence and decisions.

That balance is fragile. If many researchers receive similar recommendations from similar models, scientific writing and research choices could converge. The paper raises this risk directly, calling for plural models, varied objectives, and institutions that reward dissent and methodological diversity.

The most hopeful reading is also the most grounded: AI may help turn critique from a privilege of proximity into a more available resource. But usefulness depends on the design around it—privacy, transparency, multiple perspectives, and researchers who treat feedback as an invitation to think rather than an instruction to comply.

Science still needs human disagreement. It may also benefit from giving more humans something worthwhile to disagree with.

: Wang, B., Liang, W., Xue, J., Zhang, Y., Cao, H., Wang, D., & Yin, Y. “Human–AI Collaboration in Science at Scale: A Global Large-scale Randomized Field Experiment.” arXiv:2605.24180v1, May 22, 2026. https://arxiv.org/pdf/2605.24180 : arXiv record for 2605.24180, accessed October 20, 2026. https://arxiv.org/abs/2605.24180 : Academy of Management Proceedings abstract, DOI 10.5465/AMPROC.2026.21478abstract, accessed October 20, 2026. https://doi.org/10.5465/AMPROC.2026.21478abstract

Research references: [1] [2] [3]