Pairwise Preferences between Summaries, Sentences, and n-Grams for Automatic Text Summarization

Abstract:

Automatic text summarization aims at reducing the length of given input documents while preserving the most important content. Both automatic generation of summaries and automatic evaluation of summarization systems are unsolved problems. In this talk, we will first discuss why we should use pairwise preferences between summaries to evaluate automatic evaluation systems instead of computing Pearson correlation scores with human judgments. Second, we will see that achieving state-of-the-art automatic evaluation performance does not require expensive reference summaries but only simple pairwise preferences between sentences. Third, we build a summarization system based on pairwise preferences between n-grams which learns to estimate information importance without the help of document-derived importance signals. I argue that learning to estimate information importance without document-derived importance signals is a crucial step towards human-level summarization capabilities.