How Good Is Amazon Translate?
By: Chris Hare
Tags: Translation, Machine Learning, AWS, Artificial Intelligence

As part of my Medium article An Overview of Amazon Translate, one of the questions is how good is Amazon Translate? It isn’t possible without a lot of time to check all 55 supported languages, so I am focusing on

English to Spanish;

Spanish to English;

English to French; and,

French to English.

I used one of the paragraphs in this article for this text. The paragraph reads:

"Another example is real-time translation for support and help-desk services. Users are not limited by their knowledge of the language your corporation uses. A Spanish speaking user (for example) can type in Spanish, the text is translated to English using Amazon Translate and displayed to the support agent. The support agent can then respond in English and the text is then translated to Spanish. Any text communication could be processed using Amazon Translate before sending it to the desired audience."

This text was translated to both French and Spanish and provided the reviewers.

English to French

The French text as translated by Amazon Translate reads:

"Un autre exemple est la traduction en temps réel pour les services d'assistance et d'assistance. Les utilisateurs ne sont pas limités par leur connaissance de la langue utilisée par votre entreprise. Un utilisateur parlant espagnol (par exemple) peut taper en espagnol, le texte est traduit en anglais à l'aide d'Amazon Translate et affiché à l'agent de support. L'agent de support peut alors répondre en anglais et le texte est ensuite traduit en espagnol. Toute communication textuelle peut être traitée à l'aide d'Amazon Translate avant de l'envoyer à l'audience souhaitée."

This text was provided to the translator to translate to English. This way we can compare how close the translation is to the original English text.

The translated text was provided to the reviewer who provided this English translation:

"Another example is translation in real time for the assistance services and assistance. Users are not limited by their knowledge of the language used by your business. A user that speaks Spanish (for example) can type in Spanish, the text is translated to English with the help of Amazon Translate and displayed to the support agent. The support agent can then respond in English and the text is then translated to Spanish. All text communication can be processed with the help of Amazon Translate before being sent to the desired audience."

If we create a visual comparison of the original and translated English, we can see the differences.

Comparing the English to French Translation

If you read both the original and the translated versions, you will see they read about the same, despite the highlighted differences. The translator had the following observation about the English to French translation:

“I find that the translation service did quite a good job. In terms of general understanding, it translated the meaning of the English passage well and a French speaker is able to understand the general idea of the paragraph and know what is being talked about. Certain technical terms (like assistance services which I’m assuming was meant to be Help Desk?) did not translate correctly. Usually in French we use the term “centre de soutien” or “centre d’assistance” is also used. Other than this, I found that the translation was quite accurate and easy to understand.” [1]

This highlights an important point. When writing to translate into a different language, it is important to be clear and concise. It is also important to validate the translation of specific terms like “help-desk”, can translate to another language, but may not be as effective as using the term appropriate for that language (which is where the Custom Terminology feature of Amazon Translate can be valuable).

French to English

This test involves having the reviewer create a sample paragraph in their language and provide the English translation. The source language text was translated into English and compared against the provided translation.

The provided French text is:

Bonjour, j’ai acheté un abonnement à votre site web la semaine passé et j’ai quelques problèmes avec mon compte. Je ne me souviens pas de mon mot de passe. Existe-t-il un moyen de réinitialiser mon mot de passe et de récupérer accès à mon compte?

After processing this text through Amazon Translate, the English text reads:

"Hello, I bought a subscription to your website last week and I have some problems with my account. I don't remember my password. Is there a way to reset my password and recover access to my account?"

The English version provided by the translator is:

"Hello, I bought a subscription to your website last week and I have some problems with my account. I do not remember my password. Is there a way to reset my password and regain access to my account?"

There is a very high degree of similarity between these two translations. The visual comparison showing the highlighted differences is here.

Comparing the French to English Translation

The differences between the two are almost insignificant. Using “don’t” instead of “do not”, and “recover” instead of “regain” are good examples of how the translation can be different, but the actual communication would be very efficient.

English to Spanish

After processing by Amazon Translate the Spanish text reads:

Otro ejemplo es la traducción en tiempo real para servicios de soporte y asistencia. Los usuarios no están limitados por su conocimiento del idioma que usa su empresa. Un usuario que habla español (por ejemplo) puede escribir en español, el texto se traduce al inglés mediante Amazon Translate y se muestra al agente de soporte. El soporte puede responder en inglés y el texto se traduce al español. Cualquier comunicación de texto podría procesarse con amazon Translate antes de enviarla a la audiencia deseada.

I sent the translated text to the translator for her to translate back to English. Her translation reads as:

Another example is real-time translation for support and assistance services. Users are not limited by their knowledge of the language your company uses. For example, a Spanish speaking user can write in Spanish, the text is translated to English using Amazon Translate and shown to the support agent. The support agent can respond in English and the text is then translated into Spanish. Any text communication could be processed via Amazon Translate before sending it to the desired audience.

If we put the two English paragraphs beside each other, we can highlight the words which differ.

Differences between Amazon Translate and the Translator

There are a few differences between the original English version and the translation from Spanish as indicated by the highlighted words. The translated text has 5 different words, 1 in the wrong place (“for example”), and one word which existed in the original text, but missing from the translation.

Spanish to English

This test involves having the translator create a sample paragraph in their language and provide the English translation. The source language text was translated into English and compared against the provided translation.

The translator provided this Spanish text for processing through Amazon Translate:

Como parte de sus sesiones temáticas, el CIDI definió temas estratégicos que incluyeron experiencias de alianzas público privadas; los desafíos y las oportunidades de alcanzar una educación de calidad; experiencias para reducir la pobreza y promover la inclusión social; la implementación del Marco de Sendai y del Plan Regional en la Reducción del Riesgo de Desastres; y el papel que puede jugar la OEA para apoyar a los Estados a alcanzar una energía asequible, confiable y sustentable. Asimismo, se exploraron alianzas con la Fundación Panamericana para el Desarrollo y la Fundación para las Américas.

The English text after translation is:

As part of its thematic sessions, CIDI defined strategic themes that included experiences of private public partnerships; challenges and opportunities to achieve quality education; experiences to reduce poverty and promote social inclusion; implementation of the Sendai Framework and the Regional Plan for Reducing the Disaster Risk; and the role that the OAS can play in supporting States to achieve affordable, reliable and sustainable energy. Partnerships were also explored with the Pan American Development Foundation and the Foundation for the Americas.

I provided the English text back to Victoria for comparison with her translation. The translator’s version of the Spanish paragraph provided is:

As part of its thematic meetings, CIDI identified strategic topics, which included experiences with public-private partnerships; challenges and opportunities in ensuring quality education; experiences in poverty reduction and fostering social inclusion; implementation of the Sendai Framework and the Regional Plan of Action for Disaster Risk Reduction; and the potential role of the OAS in assisting states in ensuring access to affordable, reliable, and sustainable energy. Partnerships were also explored with the Pan American Development Foundation and the Trust for the Americas.

If we put the two English paragraphs beside each other, we can highlight the words which differ.

Differences between Amazon Translate and the Translator

It looks from the highlighted words like Amazon Translate didn’t do a great job at translating the sample paragraph from Spanish to English. There are 22 different words in the translated Spanish than what was provided by the translator. If you read the translator’s version first and then read the Amazon Translate version, you will see they are closer in meaning than it would appear.

Conclusion

Amazon Translate did pretty well in the translation samples we examined. This was by no means an exhaustive study of the capabilities of amazon Translate. The intent was to demonstrate that while there are syntactic differences in the translated text, the message appears to be presented.

That being said, whether or not to use Amazon Translate for any specific project should depend on your unique situation. These factors should be considered in your decision:

When you need near real-time translation, it is appropriate to use Amazon Translate. When 100% accuracy is less important than ”getting the point across”, Amazon Translate makes sense. For long, complex documents where accuracy is more important, the use of a professional translator is more appropriate. Text involving cultural specific language, events, or history, may be more appropriate for a professional translator. Text which is ”short-lived”, such as a chat session or e-mail, is suitable for Amazon Translate. It is important to consider that issues could arise because people may not always type what they mean. Text which is intended for storage in an archival system or used for reference by others may benefit from a professional translator. Be careful of redundant phrases. Be clear and concise. Validate the translation of specific terms like “help-desk”, can translate to another language, but may not be as effective as using the term appropriate for that language.

Acknowledgments

Thank you to Chanelle Dupuis, for her assistance with the English-French part of this evaluation. Ms. Dupuis is currently working toward her Ph.D. in French Studies at Brown University, and is fluent in French, Spanish, Portuguese, and English.

Thank you to Victoria Martinez Adalid, for her assistance with the English-Spanish part of the evaluation. Ms. Adalid is a Professional English-Spanish Translator.

The contributions made by these professionals helped greatly with the content in this research.

References

[1]: Email Conversation with Chanelle Dupuis, July 2020.

A Deep Dive into Amazon Polly

A Five Overview of AWS Transcribe

Amazon Comprehend

Amazon Polly: Bringing Audio to my Medium Articles

Amazon Translate Overview

Introduction to Amazon Translate Digital Training

What is Machine Translation? Rule Based Machine Translation vs. Statistical Machine Translation

About the Author

Chris is a highly-skilled Information Technology, AWS Cloud, Training and Security Professional bringing cloud, security, training, and process engineering leadership to simplify and deliver high-quality products. He is the co-author of seven books and author of more than 70 articles and book chapters in technical, management, and information security publications. His extensive technology, information security, and training experience make him a key resource who can help companies through technical challenges.

Copyright

This article is Copyright © 2020, Chris Hare.