Potential and limits of using post-edits as reference translations for MT evaluation
MetadataShow full item record
This item's downloads: 89 (view details)
Popovic, Maja, Arcan, Mihael, & Lommel, Arle. (2016). Potential and limits of using post-edits as reference translations for MT evaluation. Paper presented at the EAMT 2016 The 19th Annual Conference of the European Association for Machine Translation, Riga, Latvia, 30 May - 01 June.
This work investigates the potential use of post-edited machine translation (MT) outputs as reference translations for automatic machine translation evaluation, focusing mainly on the following important question: Is it necessary to take into account the machine translation system and the source language from which the given post-edits are generated? In order to explore this, we investigated the use of post-edits originating from different machine translation systems (two statistical systems and two rule-based systems), as well as the use of post-edits originating from two different source languages (English and German). The obtained results shown that for comparison of different systems using automatic evaluation metrics, a good option is to use a post-edit originating from a high-quality (possibly distinct) system. A better option is to use it together with other references and post-edits, however post-edits originating from poor translation systems should be avoided. For tuning or development of a particular system, post-edited output of this same system seems to be the best reference translation.