Impact of large language models on peer review opinions from a fine-grained perspective: Evidence from top conference proceedings in AI

The study's value isn't in confirming that LLMs affect writing style (that's been assumed for two years) but in whether they're changing what reviewers actually reward: the evaluative criteria, the recommendation thresholds, the things that get papers accepted or rejected at the venues that set the field's agenda.

This connects directly to Modelwire's recent coverage of LLM judge reliability. The 'Diagnosing LLM Judge Reliability' piece from April 16 found that even when aggregate consistency looks high, a substantial fraction of individual judgments are logically inconsistent. If reviewers are now using LLMs to help form or articulate opinions, those same failure modes propagate into peer review. The 'Context Over Content' story from the same day adds another layer: LLM judges demonstrably shift their verdicts based on stakes framing, which is exactly the kind of bias that would be hard to detect in aggregate conference-level data.

Watch whether the authors release venue-level breakdowns showing which conferences show the strongest signal. If the effect concentrates in venues that adopted LLM-assisted review tooling earliest, that's evidence of a direct mechanism rather than a diffuse cultural shift.