Publications

What is a Publication?
69 Publications visible to you, out of a total of 69

Abstract

Not specified

Authors: Wei Liu, Stephen Wan, Michael Strube

Date Published: 16th Jun 2024

Publication Type: Journal

Abstract

Not specified

Authors: Sungho Jeon, Ching-Feng Yeh, Hakan Inan, Wei-Ning Hsu, Rashi Rungta, Yashar Mehdad, Daniel Bikel

Date Published: 14th Apr 2024

Publication Type: InProceedings

Abstract

Not specified

Authors: Xianghe Ma, Michael Strube, Wei Zhao

Date Published: 17th Mar 2024

Publication Type: InProceedings

Abstract (Expand)

Unlike classical lexical overlap metrics such as BLEU, most current evaluation metrics for machine translation (for example, COMET or BERTScore) are based on black-box large language models. They often achieve strong correlations with human judgments, but recent research indicates that the lower-quality classical metrics remain dominant, one of the potential reasons being that their decision processes are more transparent. To foster more widespread acceptance of novel high-quality metrics, explainability thus becomes crucial. In this concept paper, we identify key properties as well as key goals of explainable machine translation metrics and provide a comprehensive synthesis of recent techniques, relating them to our established goals and properties. In this context, we also discuss the latest state-of-the-art approaches to explainable metrics based on generative models such as ChatGPT and GPT4. Finally, we contribute a vision of next-generation approaches, including natural language explanations. We hope that our work can help catalyze and guide future research on explainable evaluation metrics and, mediately, also contribute to better and more transparent machine translation systems.

Authors: Christoph Leiter, Piyawat Lertvittayakumjorn, Marina Fomicheva, Wei Zhao, Yang Gao, Steffen Eger

Date Published: 1st Mar 2024

Publication Type: Journal

Abstract

Not specified

Authors: Haixia Chai, Michael Strube

Date Published: 6th Dec 2023

Publication Type: InProceedings

Abstract

Not specified

Authors: Mehwish Fatima, Tim Kolber, Katja Markert, Michael Strube

Date Published: 6th Dec 2023

Publication Type: InProceedings

Abstract

Not specified

Authors: Wei Zhao, Federico López, J. Maxwell Riestenberg, Michael Strube, Diaaeldin Taha, Steve Trettel

Date Published: 18th Sep 2023

Publication Type: InProceedings

Powered by
(v.1.14.2)
Copyright © 2008 - 2023 The University of Manchester and HITS gGmbH