Merchants of MT “Equivalent” to Human Translation Rely on Artful Lies and Gullibility of Their Customers

“What translators are saying is that machine translation should be forbidden and only human translations should be allowed,” said a commenter on a Facebook group of translators recently.

It should be noted that when I took a look at who this person was, I saw that, purely coincidentally, this person who infiltrated a number of groups for translators although he himself is not a translator, was selling his own customized machine translation system.

What he said was of course a lie. That is not what translators are saying. At least not translators who still have a functioning brain. Only a total moron could believe that anybody has the power to “forbid machine translations”, especially since most machine translations are free. Not even translators are probably as stupid as the above-mentioned troll suggested.

What we are saying is that machine translation is only a tool and not translation.

We understand that machine translations are very useful, not only to translators, but also and especially to non-translators. The problem is that people are led to believe by the “translation industry” that machine translations are actual translations, because they are constantly bombarded by insidious propaganda of the “translation industry” whose aim is to erase in the minds of most people the difference between human translations and machine translations so that the machine translation snake oil could then be sold to clients as real translations.

After all, machine translation is so much cheaper than actual human translation!

Machine translations may sometime look like real, i.e. human translations, which happens once in a while when the stars are aligned just right, which is to say when an algorithm found a previous translation of a human translator that happens to be perfectly compatible with a document “translated” by the machine translation tool. But as anybody who has dealt with machine translations knows, looks can be deceiving.

“Human Parity of Machine Translation” Is a Myth

According to a recent article published on September 7, 2018, by Gino Diño in Slator, which criticizes among other things the transparent hype created around “neural machine translation,” so-called human parity of machine translation is not what the words seem to suggest, as is often the case when we talk about misinformation launched into the world by the “translation industry.”

As the same Slator reporter wrote in a previous article titled “In Human vs. Machine Translation, Compare Documents, Not Sentences,” Microsoft authors claim in their paper that human parity is achieved “if there is no statistically significant difference between human quality scores for a test set of candidate translations from a machine translation system and the scores for the corresponding human translations. In other words, if a bilingual human evaluator judges the quality of human and machine translations as equal (difference in scores are statistically insignificant), then the machine has achieved human parity.”

On surface, such a claim appears to be reasonable. But as an English politician put it a long time ago, “there are lies, damned lies, and statistics.” When a group of independent researchers, Läubli, Hennrich and Volk, from Standford University, University of Edinburgh, and Zurich University, respectively, who were not paid for their research by Microsoft, looked more closely at Microsoft’s human parity claim, they found some very interesting facts:

“Microsoft followed current research standards in their methodology, where usually, “raters see single sentences – one by one, from any of the test documents, in random order – and rate their adequacy and fluency on a scale from 0 to 100.”

However, in this process, Läubli said it would be “impossible” for evaluators to detect certain translation errors, and thus they were unable to properly take these into account.

He pointed out some of the major problems in Microsoft’s process, among others:

1. Evaluators were bilingual crowd workers, not necessarily professional translators.
2. Evaluators only assessed adequacy, not fluency.
3. Evaluators “never directly compared human to machine translation.” They looked at them separately and assigned scores.”

It’s no surprise to me that these evaluators reached the conclusion that Microsoft wanted them to reach. A translation that purposely ignores context is not a translation. Such a translation is bound to result in major, unforgivable and meaning-changing errors because context is the oxygen without which the meaning of the words, the sentences and of the entire document suffocates and dies.

Läubli, Hennrich and Volk further write that the NMT evaluation methods need to be changed, and even state that “Spreading rumours about human parity is dangerous for both research and practice: funding agencies may not want to fund MT research anymore if they think that the problem is “solved” and translation managers are not going to be willing anymore to have professionals revise MT output at all.”

Let’s say that a well researched scientific paper of several thousand words that is based on the Microsoft style statistical method for machine translation results evaluates the prospect of mankind to survive World War III in a document that is translated from Chinese into English. Even if the conclusion of the article in Chinese were to be that humans WOULD NOT survive such a catastrophic event, based on the statistical evaluation method that compares only words in separate sentences, without paying attention to context (since the meaning of the context is something that cannot be programmed into an algorithm), the result of the machine-translated version could be easily that humanity WOULD survive such an event.

A mistake like this can happen very easily in a machine translation. I have seen it dozens of times in machine translations of German patents when the word no (nicht), which can be in German hidden at the end of a long German sentence where the verb is sometime found, was wrongly assigned or missed by the software, although such a mistake would not be missed by a human translator who understands and pays attention to the context.

Machine programs for Japanese suffer from a similar problem with verbs that can change the meaning of an entire paragraph and are hidden in places where algorithms fail to find them, as well as with continuous series of characters which are often interpreted erroneously, just like compound nouns are often misinterpreted by super-cool MT algorithms in German.

In fact, the more translators use computer tools, for example to ensure consistent terms, the more their translations may be exposed to grave and really stupid errors caused by non-thinking algorithms. Just yesterday I was proofreading an excellent translation of a patent into German in which there were many “compound nouns” containing prepositions and articles that were run together and joined with nouns. After talking to the translator, I discovered that the problem, that was initially invisible to the translator in MS Word, was probably caused by the CAT tool that he was using (the much beloved and celebrated memo-Q in this case).

IT that Helps People Work Better and IT that Replaces People

There is a difference between IT that helps people to work better and IT that replaces them. Computers and IT have been affecting the work of all of us for several decades now, regardless of what we do for living. There have been many articles claiming that most doctors and lawyers will be replaced in the next few years by IT as some professions have been already, and it is likely that, for example, an anesthesiologist could be replaced by a specialized computer as described in this Washington Post article which is already three years old.

After all, a state-of-the-art machine is much cheaper than a doctor!

But would you want to be diagnosed by a machine before an operation that could (or not if it is based on incorrect information) save your own life?

I would want the anesthesiologist to have the best equipment with the best software currently available to do the job. But I would not want to have a machine diagnose me before an operation instead of a doctor because I happen to think that the chances that replacing an experienced and highly educated human called anesthesiologist by state-of-the-art equipment with the best available software would kill me are unacceptably high.

The chances of replacing an experienced and highly educated human translator with a tool that most translators have been using for quite some time, called machine translation, will kill the meaning of the translation are equally high.

But that is not how the “translation industry” looks at machine translation and what it can and cannot do, because the industry is only interested in maximizing its profit.

So it is only natural that the industry would design an evaluation method according to which “bilingual crowd workers”, whatever that means, compare human translation to machine translation by looking only at separate sentences and assigning points to them without looking at the context or at the meaning of the entire article and without actually comparing the human translation to the MT result, so that they (“the bilingual crowd workers” who are not translators and who don’t consider the meaning of the entire document) would find MT to be “equivalent” to human translation.

First you design a method that is guaranteed to prove your point and then you publish the result of your impartial test with great fanfare.

After all, that is how the “translation industry” has been selling its snake oil to gullible customers for quite some time now.