Pages in topic:   < [1 2 3 4] >
Your thoughts on the quality of "premium" Google Translate?
Thread poster: Samuel Murray
Samuel Murray
Samuel Murray  Identity Verified
Netherlands
Local time: 12:27
Member (2006)
English to Afrikaans
+ ...
TOPIC STARTER
French-English literary translation Dec 20, 2016

ph-b wrote:
I don’t know if and to what extent the translations are representative of what Google Cloud Translation can do with a not too technical text, and I’d like to see how it would handle a contract, for instance, or indeed a literary text...


I had a few minutes to spare, so I found a reasonably modern French text that has (I assume) a good English translation. Unfortunately I had to OCR the French text, but Google Premium was able to look past that.

You can download a somewhat longer sample, but here is a sneak peek:

FR-EN compare literary translation

These opening sentences might make it appear as if the Premium translation isn't much better than the standard one, but take a look at the download file and tell us your opinion.

Samuel

PS: Com- punction... well, that's what you get with PDF.

[Edited at 2016-12-20 10:27 GMT]


 
dkfmmuc
dkfmmuc  Identity Verified
Local time: 12:27
Member (2009)
German to English
+ ...
Objective of this thread Dec 20, 2016

Dear Samuel,

just have read the whole thread and I am not sure about the objective respectively the overall statement!

Therefore please allow me just to ask about the objective of this thread:

1.) Do you think that many companies will use that und than the [human] translators will get more proofreading
and less translation orders?
2.) Do you think that this is a fascinating technology or a threat?
3.) Do you think that this google "servi
... See more
Dear Samuel,

just have read the whole thread and I am not sure about the objective respectively the overall statement!

Therefore please allow me just to ask about the objective of this thread:

1.) Do you think that many companies will use that und than the [human] translators will get more proofreading
and less translation orders?
2.) Do you think that this is a fascinating technology or a threat?
3.) Do you think that this google "service" has a competitive edge compared to the other solutions on the market?

Info would be fine.

Best regards

Gerd

[Edited at 2016-12-20 10:48 GMT]

[Edited at 2016-12-20 10:57 GMT]

[Edited at 2016-12-20 10:58 GMT]
Collapse


 
Samuel Murray
Samuel Murray  Identity Verified
Netherlands
Local time: 12:27
Member (2006)
English to Afrikaans
+ ...
TOPIC STARTER
@Gerd Dec 20, 2016

dkfmmuc wrote:
Therefore please allow me just to ask about the objective of this thread...


To hear opinions about the quality of the premium service from translators who speak the languages that the premium service offers. I myself can evaluate the language quality of the English target texts, but I can't evaluate the translation quality because I don't speak the source languages.

1.) Do you think that many companies will use that and than the [human] translators will get more proofreading and less translation orders?


I think that there is definitely a danger of that. I would prefer it if Google could build in some sort of clue that tells the reader that this is a machine translation (e.g. by inserting hidden characters into the text, as it currently does in some of the standard translations).

This may change the landscape a bit -- how can translators who suspect that they've been given a machine translation to proofread figure out if the translation is indeed a machine translation and not just a rather hurried human translation?

Samuel


 
dkfmmuc
dkfmmuc  Identity Verified
Local time: 12:27
Member (2009)
German to English
+ ...
Warning notice for machine translations? Dec 20, 2016

Samuel Murray wrote:

I think that there is definitely a danger of that. I would prefer it if Google could build in some sort of clue that tells the reader that this is a machine translation (e.g. by inserting hidden characters into the text, as it currently does in some of the standard translations).

This may change the landscape a bit -- how can translators who suspect that they've been given a machine translation to proofread figure out if the translation is indeed a machine translation and not just a rather hurried human translation?

Samuel


Thanks for updating and informing about the system of "hidden characters" within the machine translation. I am not really sure what I should do with that information!

When bidding for a proofreading project I normally view one page of it and then state a price. Sometimes this is a game of luck if I have
to rebuild some sentences right from the scratch. And this first page had a translation quality which is not consistent with the rest of the first version of the translation.

I am convinced that if a customer is really cost-conscious the best way to get a low-priced translation is a combination of a translator not living in a high-cost country and then assign the proofreading task to a native speaker.

The best way would be to accept that the costs of translation and fine-tuning are integral part of the business workflow and value proposition. All too often companies forget that the texts/manuals/product infos have a high impact on the decision of the end customer.

And that these parts are not cost elements but simply appreciation for the customers in your target country.





[Edited at 2016-12-20 11:22 GMT]


 
Kay-Viktor Stegemann
Kay-Viktor Stegemann
Germany
Local time: 12:27
English to German
In memoriam
First the facts, then the opinion Dec 20, 2016

dkfmmuc wrote:

Dear Samuel,

just have read the whole thread and I am not sure about the objective respectively the overall statement!

Therefore please allow me just to ask about the objective of this thread:

1.) Do you think that many companies will use that und than the [human] translators will get more proofreading
and less translation orders?
2.) Do you think that this is a fascinating technology or a threat?
3.) Do you think that this google "service" has a competitive edge compared to the other solutions on the market?


I think Samuel does it right. He does not start with an opinion, he starts with an analysis. Why should a thread like this start with a pre-formed opinion and then do the analysis later? To confirm what one thinks anyway?

It is obvious that translators should keep an eye on the latest developments in MT and that it will have an influence on the industry, one way or the other. Let's follow these developments with keen professional interest.

Regarding the English-German examples in the first post, there is definitely an improvement between standard and premium, but it is still a machine translation, this still shows in some typical mistakes like wrong choice and order of words. It is still obvious that the engine does not understand the meaning of the language and is just an imitation machine with better imitation techniques.


 
Lingua 5B
Lingua 5B  Identity Verified
Bosnia and Herzegovina
Local time: 12:27
Member (2009)
English to Croatian
+ ...
"Better" Dec 20, 2016

Kay-Viktor Stegemann wrote:
Regarding the English-German examples in the first post, there is definitely an improvement between standard and premium, but it is still a machine translation, this still shows in some typical mistakes like wrong choice and order of words. It is still obvious that the engine does not understand the meaning of the language and is just an imitation machine with better imitation techniques.


When you say there's an improvement, can you please elaborate? Is it in terms of quality or quantity of errors?

I'm asking because it seems to me that efforts, time and funds put into these "improvements" outweigh the actual outcome, ie. a huge effort for minor improvements or illusion of improvement. I'm saying illusion, because there are still mistakes that require human attention and work?

Samuel Murray wrote:

These opening sentences might make it appear as if the Premium translation isn't much better than the standard one, but take a look at the download file and tell us your opinion.

Samuel

PS: Com- punction... well, that's what you get with PDF.

[Edited at 2016-12-20 10:27 GMT]


Imagine for a second you owned a company and you had an employee who performed the same way GT does. With this task he/she may do somewhat well, tomorrow it may be a disaster - totally unpredictable but mostly bad work. What would you do with this employee, would you continue investing in them?

[Edited at 2016-12-20 16:12 GMT]


 
Kay-Viktor Stegemann
Kay-Viktor Stegemann
Germany
Local time: 12:27
English to German
In memoriam
Sample is too small Dec 21, 2016

Lingua 5B wrote:

When you say there's an improvement, can you please elaborate? Is it in terms of quality or quantity of errors?



In fact, the sample is too small to make a well founded statement here. In the first example, the difference is minimal but the premium version did not leave out part of the airplane name. In the second example, the German sentence structure in the standard version is completely broken while in the premium version the German sentence structure is correct. Also in two cases (ordered, operated) the premium version chose correct words while the standard version chose wrong translations. On the other hand, the premium version made a nonsense translation of "Australian Defense Force" while the standard version left it as it is. On the whole, I see a certain improvement here, but it would require many more samples to really have an evaluation with statistical relevance. (And no, I have not the kind of time to make such an evaluation )


 
Lingua 5B
Lingua 5B  Identity Verified
Bosnia and Herzegovina
Local time: 12:27
Member (2009)
English to Croatian
+ ...
Yes, exactly. Dec 21, 2016

Kay-Viktor Stegemann wrote:

Lingua 5B wrote:

When you say there's an improvement, can you please elaborate? Is it in terms of quality or quantity of errors?



In fact, the sample is too small to make a well founded statement here. In the first example, the difference is minimal but the premium version did not leave out part of the airplane name. In the second example, the German sentence structure in the standard version is completely broken while in the premium version the German sentence structure is correct. Also in two cases (ordered, operated) the premium version chose correct words while the standard version chose wrong translations. On the other hand, the premium version made a nonsense translation of "Australian Defense Force" while the standard version left it as it is. On the whole, I see a certain improvement here, but it would require many more samples to really have an evaluation with statistical relevance. (And no, I have not the kind of time to make such an evaluation )


Unless someone offered you min. $100 per hour for such evaluations.

And what you did here is the kind of clarity and elaboration I was looking for. Well then we can talk only on the level of this small sample where we can see the premium kind of does better at two or three aspects, then does worse on the fourth aspect. What conclusions can we draw? This works on luck and probability, like slot machines?

I only know it's a machine and their mistakes are tolerated with much, much lower criteria than humans mistakes. Mistake is a mistake, be it human or machine, it will have consequences. And it will also cause more, not less work and hassle for everyone.


 
Michael Wetzel
Michael Wetzel  Identity Verified
Germany
Local time: 12:27
German to English
English > German seems like a radical improvement, but sample size of one Dec 21, 2016

Like Kay-Viktor, I am not going to take the time to look into this. I feel justified in sticking with my preconceived notions and assuming that this is nonsense, despite some evidence to the contrary. I simply don't see it playing a role in the context of the work that I actually do.
And the whole "learning" gambit is the same nonsense they have been spouting for years. Learning requires consistent and prolonged feedback, among other things.

The German>English differences are
... See more
Like Kay-Viktor, I am not going to take the time to look into this. I feel justified in sticking with my preconceived notions and assuming that this is nonsense, despite some evidence to the contrary. I simply don't see it playing a role in the context of the work that I actually do.
And the whole "learning" gambit is the same nonsense they have been spouting for years. Learning requires consistent and prolonged feedback, among other things.

The German>English differences are insubstantial, but the English>German difference seems radical: The standard GT translation is useless gibberish, while premium GT has produced an accurate translation with an error (translation of a proper name that should not have been translated) and a structure that is awkward, but correct and comprehensible.

I would be interested in seeing what premium GT does with real texts from articles in journals or magazines and newspapers, where less effort (or no effort) has been made to write simply, and GT needs to deal with normal sentences of 20-40 words. I simply assume that it cannot consistently effectively deal with that.

The funny thing is that many translators often seem just as susceptible to Google's marketing machine as the lay people mentioned by Preston ...

jnagel1000 wrote:

My impression is that the AI analzyes the whole document and then makes the translation.

In short, the entire industry will be obsolete quite soon.

AI is already superintelligent and is getting smarter at an exponential rate.



PS in response to Lingua 5B: This sample of one(!) complex sentence does seem to suggest a systematic and fundamental improvement and not a chance result or a zero-sum game of trade-offs. I strongly disagree with that assessment.
However, I do agree with your general assessment that it would be silly to suspend our disbelief for another five years, waiting for the end of the (translation) world.


[Edited at 2016-12-21 10:26 GMT]
Collapse


 
Lingua 5B
Lingua 5B  Identity Verified
Bosnia and Herzegovina
Local time: 12:27
Member (2009)
English to Croatian
+ ...
Superinteligent GT. Dec 21, 2016

If it's super-intelligent, why can't it read? If it could read, it could write properly too. Reading and writing are two sides of the same coin.

Instead, it just analyzes and provides (wild) "guesses". The Era of Robots.


[Edited at 2016-12-21 10:25 GMT]


 
ph-b (X)
ph-b (X)
France
Local time: 12:27
English to French
+ ...
Not my language pair. Dec 21, 2016

Samuel Murray wrote:

I found a reasonably modern French text that has (I assume) a good English translation.]


Thank you for taking the trouble, Samuel, but I don't translate into English, so I'd rather not assess the quality of the translation or of the target language.

[Edited at 2016-12-21 21:21 GMT]


 
Samuel Murray
Samuel Murray  Identity Verified
Netherlands
Local time: 12:27
Member (2006)
English to Afrikaans
+ ...
TOPIC STARTER
Long sentences: English-German Dec 21, 2016

Michael Wetzel wrote:
I would be interested in seeing what premium GT does with real texts from articles in journals or magazines and newspapers, where less effort (or no effort) has been made to write simply, and GT needs to deal with normal sentences of 20-40 words.


If you give me a URL to some journal/magazine article, I can generate a preview.

In the mean time, here are three English sentences that are just under 100 words long, translated into German.

long german 1

long german 2

long german 3


 
Samuel Murray
Samuel Murray  Identity Verified
Netherlands
Local time: 12:27
Member (2006)
English to Afrikaans
+ ...
TOPIC STARTER
Odd sentences: German-English Dec 21, 2016

Samuel Murray wrote:
In the mean time, here are three English sentences that are just under 100 words long, translated into German.


I could not find a similar resource for German-English, but this page had some curiosities on it. I'm sure the first sentence's "premium" translation comes straight out of a TM and that there is nothing neural about it. If I understand correctly, the 4th sentence is old German, so perhaps it doesn't count either.

long english 1

long english 2


[Edited at 2016-12-21 21:50 GMT]


 
Kay-Viktor Stegemann
Kay-Viktor Stegemann
Germany
Local time: 12:27
English to German
In memoriam
Standard garbage and premium garbage Dec 21, 2016

Samuel Murray wrote:

In the mean time, here are three English sentences that are just under 100 words long, translated into German.


After a short scan of these EN>DE translations, my assessment would be that both variants are garbage, though different garbage. In these examples I cannot see any significant improvement.

On the other hand, the sample sentences you chose are rather extraordinary, long-winded literary sentences. Maybe this is not representative for the average translation jobs we are facing every day. What about product descriptions, user manuals, software strings? Everyday stuff?


 
Samuel Murray
Samuel Murray  Identity Verified
Netherlands
Local time: 12:27
Member (2006)
English to Afrikaans
+ ...
TOPIC STARTER
@Kay Dec 21, 2016

Kay-Viktor Stegemann wrote:
The sample sentences you chose are rather extraordinary, long-winded literary sentences.


Yes, but that was deliberate, since one member posited that while the premium service does fairly well on short sentences, it may not do well on longer, more complicated sentences.


 
Pages in topic:   < [1 2 3 4] >


To report site rules violations or get help, contact a site moderator:


You can also contact site staff by submitting a support request »

Your thoughts on the quality of "premium" Google Translate?






CafeTran Espresso
You've never met a CAT tool this clever!

Translate faster & easier, using a sophisticated CAT tool built by a translator / developer. Accept jobs from clients who use Trados, MemoQ, Wordfast & major CAT tools. Download and start using CafeTran Espresso -- for free

Buy now! »
Protemos translation business management system
Create your account in minutes, and start working! 3-month trial for agencies, and free for freelancers!

The system lets you keep client/vendor database, with contacts and rates, manage projects and assign jobs to vendors, issue invoices, track payments, store and manage project files, generate business reports on turnover profit per client/manager etc.

More info »