Findings of the Fifth Shared Task on Multilingual Coreference Resolution: Expanding Datasets for Long-Range Entities

The shared task explicitly targets long-range entity chains as a distinct evaluation problem, not just incremental dataset expansion. This signals that coreference resolution has matured enough to isolate and measure a specific failure mode: models that handle local mention clustering but collapse on entities separated by many sentences.

This benchmark work sits alongside recent mechanistic research into how transformers process language. The conditional scale entropy paper from May showed how models resolve semantic divergence across layers; this coreference task now provides a structured evaluation of whether that internal machinery actually preserves entity identity over document-length distances. The four LLM-based systems competing here will reveal whether current architectures have genuinely solved long-context understanding or merely appear to via retrieval shortcuts. That distinction matters for the multilingual robustness findings from the syncretism study, which validated that LLMs capture real cross-linguistic phenomena. If long-range coreference fails systematically across languages, it suggests the phenomena capture is shallow.

If the LLM-based systems outperform traditional mention-clustering baselines on the long-range subset but not the local subset, that confirms models are using position-aware attention rather than genuine entity tracking. Conversely, if performance gaps persist equally across both subsets, the benchmark has isolated a genuine architectural limitation worth targeting in future pretraining or fine-tuning work.