Please use this identifier to cite or link to this item:
http://drsr.daiict.ac.in//handle/123456789/1122
Title: | Finding Proxy For Human Evaluation Re-evaluating the evaluation of news summarization |
Authors: | Majumder, Prasenjit Ranpara, Tarang J. |
Keywords: | News Summarization Evaluation Lexical overlap Contextual Similarity ROUGE Transformers Uord2vec |
Issue Date: | 2022 |
Publisher: | Dhirubhai Ambani Institute of Information and Communication Technology |
Citation: | Ranpara, Tarang J. (2022). Finding Proxy For Human Evaluation Re-evaluating the evaluation of news summarization. Dhirubhai Ambani Institute of Information and Communication Technology. xi, 51 p. (Acc. # T01042). |
Abstract: | Engaging human annotators to evaluate every summary in a content summarization system is not feasible. Automatic evaluation metrics act as a proxy for human evaluation. A high correlation with human evaluation determines the effectiveness of a given metric. This thesis compares 40 different evaluation metrics with human judgments in terms of correlation and investigates whether the contextual similarity based metrics are better than lexical overlap based metrics, i.e., ROUGE score. The comparison shows that contextual similarity based metrics have a high correlation with human judgments than lexical overlap based metrics. Thus, such metrics can act as a good proxy for human judgment. |
URI: | http://drsr.daiict.ac.in//handle/123456789/1122 |
Appears in Collections: | M Tech Dissertations |
Files in This Item:
File | Size | Format | |
---|---|---|---|
202011057.pdf | 1.32 MB | Adobe PDF | View/Open |
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.