Posted by: patenttranslator | May 13, 2016

The Case Against Blind Tasting of Translations by “The Translation Industry”

Drunk Monk

In a blind wine tasting that took place forty years ago in France and later became known somewhat ironically as “The Judgment of Paris”, the underdog, California Chardonnays and Cabernet Sauvignons defeated the undisputed king of the best wines in the entire world up until that point, namely France’s Burgundy and Bordeaux vines.

I remember that 20 years ago on the 20th anniversary of that historical victory of America over France, I saw on my local TV station a program celebrating this major victory of the California wine industry (everything’s an industry now, including charity and religion). The commentators and witnesses of the event on TV were still, 20 years later, somewhat giddy over the historic victory over France.

They were talking about French oenologists who were sipping wines from unlabeled bottles and then spitting them out (which I find horrifying and barbaric) and who were saying things like “now we’re talking”, in French of course, because they thought they were tasting French wines. At the time I was living in Santa Rosa, California, which is located in Sonoma Wine Country and was able to visit over a period of almost a decade quite a few wineries in Sonoma and Napa Wine Country. I was feeling appropriate patriotic pride about the victory of California wines two decades ago, although I was only a relatively recent Californian, having lived there since 1982.

The French wine experts who gave the first prize to Napa wines, (while convinced that these wines were from France), were guided by their sophisticated taste buds, sense of smell, and experience of many years. They might have been wrong about the origin of the wines, but in their judgment they were right about the most important thing: the quality and taste of the heavenly libations.

You don’t really have to be a specially trained wine industry expert to appreciate the difference of quality of different wines. Anybody can tell the difference between two different wines. I remember that shortly after the fall of the Berlin Wall 27 years ago, the bums that one could see drinking wine and beer since the morning in downtown San Francisco switched for a while from big bottles of cheap, fortified California wines to a sweet Bulgarian table wine called Mecha Krv (Bear Blood) in Bulgarian.

Bear Blood was sold only in smaller bottles and has a lower alcohol content, but it was clearly a superior wine for about the same price as far as the bums were concerned.

The blind tasting that takes place every day hundreds of times in “the translation industry” is organized somewhat differently, but has basically the same purpose as wine tasting: to try to ascertain the quality of translations.

I am talking about obligatory translation tests that are administered en mass by translation agencies, especially but not only by large translation agencies, to translators or would-be translators who want to work for these agencies.

Although these tests are now accepted by many but not all translators as a necessary ritual, or hoop to be jumped through, two or three decades ago this was not the case. This is because two or three decades ago, there were very few large translation agencies claiming that they specialized in everything and that they were able to translate equally well “all languages and all subjects”. Most translation agencies back then did in fact specialize in something, which is to say only in certain languages and subjects.

Agencies that in fact did develop expertise in a limited number of languages and subjects, nowadays often deprecatingly referred to by “the translation industry” as “single-language vendors” or something like that, did not need to send a test translation to prospective translators who are naturally expected by the “translation industry” to do these tests for free because agencies who knew their stuff were able to evaluate suitable samples of past work of prospective translators.

That is no longer the case with many of the translation agencies in “the translation industry” who automatically send a test to new prospective translators as they have no ethical problem with trying to force people to work for them for free.

Under certain circumstances, I am not necessarily against the practice of doing free tests, although I see plenty of problems with an arrangement in which people are asked to work for free as if it were the most natural thing under the sun.

When I was a beginner, I remember that I once refused as a matter of principle to do a free test in 1989 for an owner of a small translation agency in San Francisco, as I am and always have been against free labor which I consider to be an essential principle and ingredient of slavery.

But the guy called me to his office on Market Street and explained that he was bidding on a project involving translations of boxes of German documents for a law firm, and the firm had asked for a free test. So he asked me very nicely whether I would be willing to do the test while pointing out that the potential payoff could be very significant. I still thought that it was unfair, but once I was given access to relevant facts, I agreed to do the free test, the agency got the job, and the job kept myself and two other translators busy for about half a year.

Under similar circumstances, I think that a free test is defensible, although a more classy agency would pay the translator from its own funds, which has happened to me once or twice.

My impression that there are not very many classy agencies left in “the translation industry” these days has been reinforced by what I read on social media, mainly that the demand for a free, obligatory test is now very common and most translation agencies seem to require such a test as part of normal procedure.

It is clear why large translation agencies need to have these tests: since they translate all languages and subjects, project managers who handle the translations are unable to evaluate samples of prior work on their own as they cannot be expected to know all languages and all subjects. In fact, from what I read on social media, most of them don’t seem to know any language besides English, or any subject either for that matter, as most of them are young, recent college graduates who are often monolingual and who mostly work for low wages.

So, unlike the cultured French oenophiles, and unlike the thirsty bums that I used to see in downtown San Francisco decades ago, who really knew their stuff when it came to deciding which wine is better, project managers working these days for “the translation industry” have no choice but to compare an existing translation supplied to them by the agency management to test prospective translators.

In this respect, the blind testing (or blind tasting) of translations in “the translation industry” is very different from wine tasting, because the people doing the testing of translations themselves have no idea whether the test translations are good, or not so good, or pretty bad. In this respect, project managers are blind, deaf, and one could perhaps say, almost brain-dead.

If the sample translations they are using are riddled with inaccuracies, which sometimes happens, project managers will not really know about it if they don’t understand the languages. Even if they by some miracle do know both languages, would they dare to raise the issue of an inaccurate translation sample if the sample was supplied to them by their boss as a good translation? Probably not because that would mean that their boss is incompetent, which as we all know happens all the time.

All they can do is try to match the sample translations returned to them by translators to the sample translation supplied to them by management. If it is a close match, it must be a good translator. If the match is not that close, it must be a bad translator.

Although the current practice of requiring automatically free translations from prospective translators is not very reliable when it comes to establishing the quality of translations and weeding out bad translations, it is very useful and very effective from another perspective.

A translator who is willing to work for free on a sample, sometimes a very long sample, is a translator who is likely to be grateful to have any work, even work at very low rates.

This is probably one of the reasons, or perhaps the main reason, why the requirement for free labor to be provided first by the translator on a translation sample is so common, although the translation is then unlikely to be evaluated by a person who would be capable of making an evaluation of translation quality.

The testing of translators by “the translation industry” is thus a certain kind of a reality show, one where what is supposed to be real is in fact fake if you take a close look at the supposed reality. The real motive behind the show lies somewhere else.

The main problem I see with blind tastings of translations by “the translation industry” is that the people doing the tasting (or testing) of translations are unable to ascertain much from the samples.

In a singing, beauty, cooking or dancing competition, or in any other kind of competition so popular nowadays on reality TV shows, the audience is generally always qualified to judge the results, although different people may pick different winners; that is why these reality shows are so popular.

But relatively few people can tell the difference between two translations.

In the translation industry, people passing a judgment on the quality of translation tests are often unable to decide which translation is good and which is bad, especially if the agency “specializes” in all languages and subjects.

But it does not really matter too much because “the translation industry” is a different kind of a reality show in which the unstated purpose (finding pliant warm bodies willing to work for a few pennies) may be much more important than the stated one (finding a translator who can pass a translation test).

Credits: The Drunk Monk picture is from a Wikipedia entry about Blind Tasting.


  1. Dear Friend Is this assignment project for me to translate into Swahili?Yona Mwanza.


  2. Sorry, it’s not a project for translation, just a silly post.

    I hope you enjoyed it, though.


  3. There is a big difference between one of these silly, so-called routine “tests” that some agencies (especially the French for some reason) insist on, and those that a client requires to see if the translator is familiar with the subject material. I agree to do the latter, not the former. Even then, there are rules, no test must be longer than 500 words. Any agency whose “test” looks suspiciously like a real job is to be avoided.

    Liked by 2 people

  4. Thank you for this excellent article! Today I’ve read about this job offer and your blog article came into my mind. Exclusively test translations…
    Regards, Agnes Lenkey,

    Director of test translation*740C97512C076412&utm_source=LINKEDIN&utm_medium=referrer

    Oversees test translation/adaptation team and the projects assigned to this group. Manages test translation/adaptation or translation verification projects, translating test materials, reviewing translations for accuracy and appropriateness, among other tasks. Prepares written reports and maintains written correspondence with clients and contractors. Organizes and/or attends meetings and communicate with co-workers, clients, and contractors. ESSENTIAL FUNCTIONS Management Responsibilities . Leads the test translation team and supervises translation manager(s) and desktop publishing specialists on multiple projects. . Manages and supervises Test Translation Projects by allocating translators/reviewers/desktop publishing consultants, establishing and following deadlines, keeping detailed records of relevant project information, and communicating with the client and providing them with regular status reports. . Identifies tasks, estimates effort, and assigns resources working with team members. . Tracks, monitors, and communicates development progress compared to the project plan, budget, requirements, quality measures, standard process, and milestones. . Estimates time, scope, and cost of technical projects, when necessary. Test Translation/Development/Proposals . Reviews and assesses English-Spanish/Spanish-English assessment materials for language appropriateness and cultural sensitivity and bias, and ensures that there is no content variance between the original and the translation. . Translates assessment materials from English into Spanish, and vice versa. Reviews translated materials to verify appropriateness of the translation. . Reviews the language of English assessment items for the use of Plain Language, and provides suggestions to linguistically modify such items to increase language accessibility by test-takers. . Assists in project development and management as needed, both in the area of test development and test translation. . Writes proposals for responses to solicitations including government RFPs. Desktop Publishing . Oversees desktop publishing personnel in test translation projects. . Reviews layout and formatting of testing material for publication, in English or other foreign languages, to ensure that it adheres to project-specific style specifications. Client Interaction . Works with clients to start up new contracts and to resolve client-specific curriculum and assessment issues. . On occasion, advises clients, test review committee members, and other stakeholders on language assessment related issues.


  5. Thank you for your comment.

    In reality, the testing by Berlitz Languages, which is what this announcement is about, is very different from the official description.

    I described in a post two years ago how I passed an extremely difficult difficult test and became Berlitz-approved immigration court interpreter when I lived in San Francisco.


    • Fact is I didn’t have time to follow the link, sorry. For sure I should have done that before posting (I am working on a large project now); I only saw the description and somehow your article came into my mind. Do you think I should delete the link above? Maybe leave only the text? For sure I didn’t want to affirm anything negative, I was astonished by the title. Why do they call it “Director of test translation”, if it is far more than that?


      • I will delete the link if you want me to but I think it works better with the link so that people can see which company it is.

        “Why do they call it “Director of test translation”, if it is far more than that?”

        Who knows. Probably because they like stupid titles would be my guess.


      • OK, if you think so, we can leave it like it is, it is a job offer, with everything that this entails. Thank you again for your interesting blog, I enjoy every article. Thanks.


  6. So it is. Nowadays unstated slavery may be much more important than a good translation. Although it has been formally abolished in many countries, in 1861 in Russia, it is still living in genes of many people.

    Liked by 1 person

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s


%d bloggers like this: