Researchers at Dublin City University have released results of a study that suggests that, despite the hype surrounding Artificial Intelligence and Machine Translation (MT), technology struggles to complete with human expertise when translating literary texts.
The study was carried out by Dr Joss Moorkens, Assistant Professor at DCU’s School of Applied Language and Intercultural Studies, in a bid to discover how successful Machine Translation really is at translating complicated texts, such as, literary novels.
To test the efficiency, accuracy and linguistic skill of Neural Machine Translation (NMT, the most up-to-date technology used within the translation industry), and how translators use it for post-editing, a “challenging literary novel” Warbreaker, by Brandon Sanderson, was chosen as the focus.
Translators, with experience ranging from 1 – 30 years, translated the novel from English to Catalan using each the following methods: translation from scratch, and post-editing of two forms of machine translation (neural and statistical).
Explaining the study further Dr Moorkens said “Machine Translation is the use of computers to estimate how to best translate words and sentences from one language to another based on how they’ve been translated in the past. It’s quick and effective, particularly on short, clear sentences requiring a literal, word-for-word translation. However, the limitations of machine translation can be exposed when processing longer, creative sentences typically found in a literary novel, for example.”
He added that “Recent years have seen the emergence of Neural Machine Translation (NMT), a system that usually produces more fluent translations, fewer errors and less literal translations. Little if any research has been published to date on translators’ attitudes to using this new state-of-the-art software when post-editing or ‘tidying up’, particularly on challenging, literary text. Our study focuses on the translators’ perceptions of such a task, whether they found the machine translation suggestions useful, and whether they would consider post-editing literary texts in the future, and contrasts these task perceptions with our technical and temporal measurements of their activity.”
The translators taking part in the study gave feedback both before, and after the translation, through a series of interviews and pre-prepared questions.
The translation methods were ranked by speed, and all participants found that NMT software enabled them to perform their task well, however, they all stated that they preferred a translation created from scratch.
This surprising result shows that, despite NMT enabling translators to do their job faster, they felt that it limited their creativity when post-editing machine translated literature, and “conditioned” them to produce a verbatim translation, rather than an inspired one.
The study concluded that the “latest machine translation systems would be useful if they need to produce a translation “in a hurry”, but that they would not post-edit by choice as they found the process difficult and found the translations to be bland.”
State-of-the-art systems seem to be able to deal well with run-of-the-mill texts where there are unambiguous meanings, or exact terminology translations are required.
However, when it comes to literary works that have been created with imagination, using complex linguistic structures, the current Machine Translation software fails to give an intricate translation that captures the essence of the original text.
So, while one translator remarked that it was both “scary and frightening to see that the machines are getting better”, they still appear to be no match for a human translation.
Recent Comments