The Folly of the Search for Objective Metrics for Determination of Quality in Translation

“It would be nice if all of the data which sociologists require could be enumerated because then we could run them through IBM machines and draw charts as the economists do. However, not everything that can be counted counts, and not everything that counts can be counted.“

William Bruce Cameron, “Informal Sociology: A Casual Introduction to Sociological Thinking”, 1963.

It would also be nice if everything that has been translated could be measured so that a numerical or digital translation quality index would be created wherein points could be assigned for quality of translation. For instance, if the translation is understandable, it would be assigned 50 points because that is the most important aspect  of any translation.

Most machine translation system have been trying, mostly in vain, to reach this fabulous goal for about the last 70 years, so it should be worth at least 50 points. Additional points could then be assigned for things like correct grammar, original and highly idiomatic expressions, consistent terminology, or even formatting, up to a maximum of 100 points.

There is something called ISO Certification Standard, then there is also something called the EN-15038 European Quality Standard and there are probably also several other “translation quality standards” out there. But I believe that none of these standards can possibly be based on an objective evaluation of translation quality.

The fact is that the concept of translation quality will invariably mean different things to different people. A writer whose book is being translated into a foreign language will have standards for translation quality that will be very different from, for example, a patent lawyer who is using my translation to litigate a case in court or to write a new patent application.

And what is the most important additional criterion in both cases mentioned above, provided that the translation already has 50 points if it is understandable? Well, the most important criterion will be whether the client, whoever it is, likes the translation.

Some may like it, and some may not, for a number of reasons, and as some of us have learned a long time ago in Latin classes: De gustibus non disputandum est (there can be no disputes when it comes to tastes).

Which is not to say that translators should not have their own quality standards and quality metrics. They definitely should, and they should try to apply them as much as possible. But to pretend that there is an objective quality standard, or a method that can be applied to every translation, such as the “Four Eyes Principle” discussed in this post, or the misguided attempts to apply industrial standards to translation mentioned above, is in my opinion a folly. Incidentally, the word folly is a cognate of the words fool and foolish, and it is defined by the dictionary as a lack of good sense and understanding.

One should attempt to apply an objective quality standard for example to the way pharmaceuticals or parachutes are manufactured. If we don’t do that, people will die unnecessarily.

But translation is a different kind of animal, while yet another problem is that the quality of translation is often judged by people who are unable to tell a good translation from a bad one.

My customers will most of the time be able to evaluate the quality of my translation, even if they don’t read Japanese or German, because if something in the translated information does not make sense to them, it is a pretty clear indication that it is a mistranslation.

But for instance the kids who work as coordinators for translation agencies have no way of telling a good translation from a bad one – unless they understand both languages and the technical subject, which in my experience is almost never the case.

I am not an objective judge of translation quality either. Once in a long while I am simply blown away by a really beautifully done translation. But most of the time, my first impression, when I see a translation which was done by another translator for me, is negative. But I know that my subjective perception of what a translation should look like is just that, an evaluation that is by its nature very subjective.

And although I have sometime received a translation that I thought was pretty bad and I had to try to resurrect it as best as I could, I have always paid the translator the full amount of the invoice. The way I see it, I am not paying for my subjective evaluation of quality, but for the objective amount of work that the translator had to perform, regardless of what I think of the result.

One should also not forget that translators are not magicians. It is very easy to blame the translator for every problem. But it is a fact that for instance the patent applications that I have to translate are often written in a very ambiguous language that does not make a lot of sense, and although they may be even sometime riddled with mistakes in the original language, as a patent translator, I am not at liberty to rewrite the text in my translation.

As far as I can tell, the only objective metric of quality of translation is whether or not the client was satisfied with it. When I receive a check for my last translation, this counts on my subjective evaluation scale as somewhere between 50 to 100 points (I usually know whether it was closer to 50 points or 100 points, and the client probably knows it too).

And I also know that in spite of years of specialized education followed by decades of specialized translating experience in my particular field of translation, I am only as good as my last translation, just like a singer is only as good as his last song, and an actor is only as good as his last role.