Bias, Coverage, and Evaluation in Multi-Document Summarization

Abstract

When summarizing clusters of news articles covering the same event, language models often fail to capture the full range of information presented across the documents. This talk presents preliminary and planned research examining how this shortcoming may be exacerbated by cognitive biases, such as those arising from a document’s position within the cluster, its perceived authority, or structural features. These influences are investigated using simple heuristics. Furthermore, the potential of repurposing the classic Pyramid Evaluation framework is explored as a means to develop a more explainable, nuanced, and reference-free approach to summary assessment.