Posted by: patenttranslator | May 11, 2013

The Folly of the Search for Objective Metrics for Determination of Quality in Translation

“It would be nice if all of the data which sociologists require could be enumerated because then we could run them through IBM machines and draw charts as the economists do. However, not everything that can be counted counts, and not everything that counts can be counted.

William Bruce Cameron, “Informal Sociology: A Casual Introduction to Sociological Thinking”, 1963.

It would also be nice if everything that has been translated could be measured so that a numerical or digital translation quality index would be created wherein points could be assigned for quality of translation. For instance, if the translation is understandable, it would be assigned 50 points because that is the most important aspect  of any translation.

Most machine translation system have been trying, mostly in vain, to reach this fabulous goal for about the last 70 years, so it should be worth at least 50 points. Additional points could then be assigned for things like correct grammar, original and highly idiomatic expressions, consistent terminology, or even formatting, up to a maximum of 100 points.

There is something called ISO Certification Standard, then there is also something called the EN-15038 European Quality Standard and there are probably also several other “translation quality standards” out there. But I believe that none of these standards can possibly be based on an objective evaluation of translation quality.

The fact is that the concept of translation quality will invariably mean different things to different people. A writer whose book is being translated into a foreign language will have standards for translation quality that will be very different from, for example, a patent lawyer who is using my translation to litigate a case in court or to write a new patent application.

And what is the most important additional criterion in both cases mentioned above, provided that the translation already has 50 points if it is understandable? Well, the most important criterion will be whether the client, whoever it is, likes the translation.

Some may like it, and some may not, for a number of reasons, and as some of us have learned a long time ago in Latin classes: De gustibus non disputandum est (there can be no disputes when it comes to tastes).

Which is not to say that translators should not have their own quality standards and quality metrics. They definitely should, and they should try to apply them as much as possible. But to pretend that there is an objective quality standard, or a method that can be applied to every translation, such as the “Four Eyes Principle” discussed in this post, or the misguided attempts to apply industrial standards to translation mentioned above, is in my opinion a folly. Incidentally, the word folly is a cognate of the words fool and foolish, and it is defined by the dictionary as a lack of good sense and understanding.

One should attempt to apply an objective quality standard for example to the way pharmaceuticals or parachutes are manufactured. If we don’t do that, people will die unnecessarily.

But translation is a different kind of animal, while yet another problem is that the quality of translation is often judged by people who are unable to tell a good translation from a bad one.

My customers will most of the time be able to evaluate the quality of my translation, even if they don’t read Japanese or German, because if something in the translated information does not make sense to them, it is a pretty clear indication that it is a mistranslation.

But for instance the kids who work as coordinators for translation agencies have no way of telling a good translation from a bad one – unless they understand both languages and the technical subject, which in my experience is almost never the case.

I am not an objective judge of translation quality either. Once in a long while I am simply blown away by a really beautifully done translation. But most of the time, my first impression, when I see a translation which was done by another translator for me, is negative. But I know that my subjective perception of what a translation should look like is just that, an evaluation that is by its nature very subjective.

And although I have sometime received a translation that I thought was pretty bad and I had to try to resurrect it as best as I could, I have always paid the translator the full amount of the invoice. The way I see it, I am not paying for my subjective evaluation of quality, but for the objective amount of work that the translator had to perform, regardless of what I think of the result.

One should also not forget that translators are not magicians. It is very easy to blame the translator for every problem. But it is a fact that for instance the patent applications that I have to translate are often written in a very ambiguous language that does not make a lot of sense, and although they may be even sometime riddled with mistakes in the original language, as a patent translator, I am not at liberty to rewrite the text in my translation.

As far as I can tell, the only objective metric of quality of translation is whether or not the client was satisfied with it. When I receive a check for my last translation, this counts on my subjective evaluation scale as somewhere between 50 to 100 points (I usually know whether it was closer to 50 points or 100 points, and the client probably knows it too).

And I also know that in spite of years of specialized education followed by decades of specialized translating experience in my particular field of translation, I am only as good as my last translation, just like a singer is only as good as his last song, and an actor is only as good as his last role.


  1. I enjoyed the whole piece but especially the second half of the last sentence. Not because it presented a concept that was new to me but because it is one that has been with me for many, many decades and so it was like feeling a tap on the shoulder and finding a dear friend to greet again :).


  2. Wow, nobody said that to me so far, and I have thousands of comments on this blog.


  3. Steve, at the risk of sounding like a “Melanie,” I love your posts! Thank you for saying all the things many of us have thought for years, but which appear to be taboo in any ATA-sponsored gathering. IMHO the “four eyes rule” is just a way for the translation agencies — oops I mean language service companies — to market the idea that an individual professional translator cannot possibly offer the same quality that their companies can.


  4. Thank you for your comment, paris-jollies.

    1. I find “Dear Melanie” comments (“Dear Melanie, your post was totally awesome”) laughable only on other people’s blogs.

    On my blog, they are a breath of fresh air.

    2. I haven’t really thought of the famous “four-eyes rule” in this way, but I think that you are onto something here.


  5. Despite our discrepancies in viewing CATs, I tend to be with Michal.

    Of course, a translator can be just as good as his last translation. His readers, if they know who he is, are the ones who, unlike his adversaries, will remember him as a good translator definitely not just because of his last translation. They would expect to read more of his translations.

    What was the last song sung by Yves Montand? I can’t remember, but I would never forget how he sang “Les Feuilles Mortes.” And I wouldn’t forget how you explained the symptoms of Translator’s Dimentia.

    We don’t have objective metrics for translation quality. But there is something essential to survive as a translator: get the clients believing in what the translator does for them by making the clients and their clients love to read what the translator writes.


  6. […] "It would be nice if all of the data which sociologists require could be enumerated because then we could run them through IBM machines and draw charts as the economists do. However, not everything…  […]


  7. Bravo! You’ve made the point brilliantly.


  8. You’re making me blush …..


  9. […] already said what I think about the futility of universal “translation standards” in this post a few weeks ago. From what I have seen so far, I believe that attempts to design a standard that would ensure a […]


  10. […] (including mendacious claims of several layers of Acheckers@, usually from 3 to 7), Awe are ISO-certified” (which makes absolutely no sense to anybody who actually knows something…), and other bogus […]


  11. […] of industrial products) are in my opinion nothing but a mendacious advertising gimmick as I have written in several posts already. There is a proper way to type, for example, this is something that can be learned in a typing […]


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s


%d bloggers like this: