Repository: googleapis/google-cloud-python

Issue:
Translate: fails with large text (errors 411 and 413)
1. Specify the API at the beginning of the title (for example, "BigQuery: ...")
   General, Core, and Other are also allowed as types

Translate

2. OS type and version

Linux (Ubuntu 17.10)

3. Python version and virtual environment information `python --version`

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=276655

4. google-cloud-python version `pip show google-cloud`, `pip show google-<service>` or `pip freeze`

google-cloud-translate==1.3.0

5. Stacktrace if available

Here's what comes back from the API (apologies for the lack of formatting)

<html lang=en>
  <meta charset=utf-8>
  <meta name=viewport content="initial-scale=1, minimum-scale=1, width=device-width">
  <title>Error 411 (Length Required)!!1</title>
  <style>
    *{margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px}* > body{background:url(//www.google.com/images/errors/robot.png) 100% 5px no-repeat;padding-right:205px}p{margin:11px 0 22px;overflow:hidden}ins{color:#777;text-decoration:none}a img{border:0}@media screen and (max-width:772px){body{background:none;margin-top:0;max-width:none;padding-right:0}}#logo{background:url(//www.google.com/images/branding/googlelogo/1x/googlelogo_color_150x54dp.png) no-repeat;margin-left:-5px}@media only screen and (min-resolution:192dpi){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat 0% 0%/100% 100%;-moz-border-image:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) 0}}@media only screen and (-webkit-min-device-pixel-ratio:2){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat;-webkit-background-size:100% 100%}}#logo{display:inline-block;height:54px;width:150px}
  </style>
  <a href=//www.google.com/><span id=logo aria-label=Google></span></a>
  <p><b>411.</b> <ins>That’s an error.</ins>
  <p>POST requests require a <code>Content-length</code> header.  <ins>That’s all we know.</ins>

6. Steps to reproduce

See code example

7. Code examplepython 

```
# -*- coding: utf-8 -*-

import os
from google.cloud import translate

os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = "/path/to/credentials.json"

translate_client = translate.Client()

text_to_translate =     '''
總統：企業加薪 政府給獎勵
〔記者蘇永耀、陳心瑜／綜合報導〕將解決年輕人低薪問題列為今年最重要工作的蔡英文總統，她昨日指出，多年來二十二Ｋ的低薪現象，讓年輕人覺得「有沒有就業都一樣」，也影響他們積極投入就業市場的態度；今年政府將致力解決低薪，若企業的薪資結構有利於年輕人，可能參照國外作法，包括給予津貼、或企業參與政府標案時，給予一些點數上的加權等方式獎勵。
總統上週五與媒體年終茶敘時說，行政團隊今年最重要任務，就是全力改善年輕人的低薪問題；上週六總統還請幕僚到幫員工加薪的「摩斯漢堡」吃早餐。相關人士說，總統這項動作獲得企業回響，府方正考慮安排總統走訪這些企業，給予打氣及肯定。
要求民進黨智庫 研擬對策
在元旦升旗典禮後，蔡總統赴五股賀聖宮參加安座大典，她致詞表示，今年的目標是「解決低薪」，讓年輕人看得到未來，希望提高民眾的所得，讓所有人民都能過好生活。
總統在回應媒體詢問時也說，已要求民進黨中央、智庫審酌當前經濟發展趨勢，研擬解決青年及弱勢問題的對策、論述。
蔡總統表示，舉例來說，一個年輕人如果月入二十二Ｋ，父母又不在台北，每個月扣掉一萬元房租，剩下的錢能做什麼運用？就算住家裡，這樣的薪水很難買房，也就失去離家單飛機會，久而久之會讓他認為，有沒有這份工作，對他的生活沒什麼影響。
解決低薪 讓青年看到未來
總統說，相信現在很多人家裡，都有一個打不定主意要做什麼的小孩，因為兩、三萬元的月薪，會讓年輕人對未來感到焦慮；如果薪水夠好，即便工作再有挑戰性，他們也願意做。
至於如何解決這個問題，總統說，這與產業結構有關，還是要促成產業轉型，同時獎勵願意為年輕人加薪的企業。她也說，政府希望能將人力導入長照等需求大的行業，並為這些從業者加薪，她期待這樣能帶動民間企業的人力需求。
蔡總統也認為，學生最好大三、大四就能和社會接軌，而且不是做一般的臨時工作，比較好的模式是，學校教育與職業間有足夠彈性，年輕人就學後出來工作，有機會再回去唸書，希望未來教育體制提供這樣的彈性。
蔡英文總統昨日指出，今年政府將致力解決低薪，若企業的薪資結構有利於年輕人，可能參照國外作法，包括給予津貼、或企業參與政府標案時，給予一些點數上的加權等方式獎勵。圖為蔡總統昨出席總統府前元旦升旗典禮。（記者張嘉明攝）    '''

translation_result = translate_client.translate(text_to_translate,
                                                target_language='en')

print translation_result['translatedText']

```




Discussion:
I **think** the problem might be at cloud/translate_v2/client.py, line 223-234, as annotated

```
       query_params = [('target', target_language)]
       
       # this next line takes the text to be translated and converts it to UTF-8 bytes,
       # which in the case of Chinese values, can be a very long string
       query_params.extend(('q', _to_bytes(value, 'utf-8'))
                            for value in values)
        query_params.extend(('cid', cid) for cid in customization_ids)
        if format_ is not None:
            query_params.append(('format', format_))
        if source_language is not None:
            query_params.append(('source', source_language))
        if model is not None:
            query_params.append(('model', model))

        # this part passes the text to be translated in the query_params parameter, which
        # is going to be encoded into the URL that is POSTed (**not** the body of the POST).
        # Suddenly our URL (with our example code) is 8kb long.
        response = self._connection.api_request(
            method='POST', path='', query_params=query_params)

```
When
Having the same issue. any status on this?
I gave up on the Python libraries and just called the REST endpoint directly

    response = requests.post(
        url='https://translation.googleapis.com/language/translate/v2?key={}'.format(
            os.environ['TRANSLATION_API_KEY']),
        json={
            'q': list_of_things_to_translate,
            'target': 'en',
            'format': 'text'
            },
        headers={"Content-Type" : "application/json"})


Resolving patch:
diff --git a/translate/google/cloud/translate_v2/client.py b/translate/google/cloud/translate_v2/client.py
--- a/translate/google/cloud/translate_v2/client.py
+++ b/translate/google/cloud/translate_v2/client.py
@@ -17,7 +17,6 @@
 
 import six
 
-from google.cloud._helpers import _to_bytes
 from google.cloud.client import Client as BaseClient
 
 from google.cloud.translate_v2._http import Connection
@@ -129,11 +128,13 @@ def detect_language(self, values):
             single_value = True
             values = [values]
 
-        query_params = []
-        query_params.extend(('q', _to_bytes(value, 'utf-8'))
-                            for value in values)
+        data = {
+            'q': values,
+        }
+
         response = self._connection.api_request(
-            method='GET', path='/detect', query_params=query_params)
+            method='POST', path='/detect', data=data)
+
         detections = response.get('data', {}).get('detections', ())
 
         if len(values) != len(detections):
@@ -219,19 +220,17 @@ def translate(self, values, target_language=None, format_=None,
         if isinstance(customization_ids, six.string_types):
             customization_ids = [customization_ids]
 
-        query_params = [('target', target_language)]
-        query_params.extend(('q', _to_bytes(value, 'utf-8'))
-                            for value in values)
-        query_params.extend(('cid', cid) for cid in customization_ids)
-        if format_ is not None:
-            query_params.append(('format', format_))
-        if source_language is not None:
-            query_params.append(('source', source_language))
-        if model is not None:
-            query_params.append(('model', model))
+        data = {
+            'target': target_language,
+            'q': values,
+            'cid': customization_ids,
+            'format': format_,
+            'source': source_language,
+            'model': model
+        }
 
         response = self._connection.api_request(
-            method='POST', path='', query_params=query_params)
+            method='POST', path='', data=data)
 
         translations = response.get('data', {}).get('translations', ())
         if len(values) != len(translations):


Test patch:
diff --git a/translate/tests/unit/test_client.py b/translate/tests/unit/test_client.py
--- a/translate/tests/unit/test_client.py
+++ b/translate/tests/unit/test_client.py
@@ -137,12 +137,12 @@ def test_detect_language_bad_result(self):
         # Verify requested.
         self.assertEqual(len(conn._requested), 1)
         req = conn._requested[0]
-        self.assertEqual(req['method'], 'GET')
+        self.assertEqual(req['method'], 'POST')
         self.assertEqual(req['path'], '/detect')
-        query_params = [
-            ('q', value.encode('utf-8')),
-        ]
-        self.assertEqual(req['query_params'], query_params)
+        expected_data = {
+            'q': [value],
+        }
+        self.assertEqual(req['data'], expected_data)
 
     def test_detect_language_single_value(self):
         client = self._make_one(_http=object())
@@ -166,12 +166,12 @@ def test_detect_language_single_value(self):
         # Verify requested.
         self.assertEqual(len(conn._requested), 1)
         req = conn._requested[0]
-        self.assertEqual(req['method'], 'GET')
+        self.assertEqual(req['method'], 'POST')
         self.assertEqual(req['path'], '/detect')
-        query_params = [
-            ('q', value.encode('utf-8')),
-        ]
-        self.assertEqual(req['query_params'], query_params)
+        expected_data = {
+            'q': [value],
+        }
+        self.assertEqual(req['data'], expected_data)
 
     def test_detect_language_multiple_values(self):
         client = self._make_one(_http=object())
@@ -205,13 +205,12 @@ def test_detect_language_multiple_values(self):
         # Verify requested.
         self.assertEqual(len(conn._requested), 1)
         req = conn._requested[0]
-        self.assertEqual(req['method'], 'GET')
+        self.assertEqual(req['method'], 'POST')
         self.assertEqual(req['path'], '/detect')
-        query_params = [
-            ('q', value1.encode('utf-8')),
-            ('q', value2.encode('utf-8')),
-        ]
-        self.assertEqual(req['query_params'], query_params)
+        expected_data = {
+            'q': [value1, value2],
+        }
+        self.assertEqual(req['data'], expected_data)
 
     def test_detect_language_multiple_results(self):
         client = self._make_one(_http=object())
@@ -251,11 +250,15 @@ def test_translate_bad_result(self):
         req = conn._requested[0]
         self.assertEqual(req['method'], 'POST')
         self.assertEqual(req['path'], '')
-        query_params = [
-            ('target', 'en'),
-            ('q', value.encode('utf-8')),
-        ]
-        self.assertEqual(req['query_params'], query_params)
+        expected_data = {
+            'target': 'en',
+            'q': [value],
+            'cid': (),
+            'source': None,
+            'model': None,
+            'format': None,
+        }
+        self.assertEqual(req['data'], expected_data)
 
     def test_translate_defaults(self):
         client = self._make_one(_http=object())
@@ -280,11 +283,16 @@ def test_translate_defaults(self):
         req = conn._requested[0]
         self.assertEqual(req['method'], 'POST')
         self.assertEqual(req['path'], '')
-        query_params = [
-            ('target', 'en'),
-            ('q', value.encode('utf-8')),
-        ]
-        self.assertEqual(req['query_params'], query_params)
+
+        expected_data = {
+            'target': 'en',
+            'q': [value],
+            'cid': (),
+            'source': None,
+            'model': None,
+            'format': None,
+        }
+        self.assertEqual(req['data'], expected_data)
 
     def test_translate_multiple(self):
         client = self._make_one(_http=object())
@@ -315,12 +323,16 @@ def test_translate_multiple(self):
         req = conn._requested[0]
         self.assertEqual(req['method'], 'POST')
         self.assertEqual(req['path'], '')
-        query_params = [
-            ('target', 'en'),
-            ('q', value1.encode('utf-8')),
-            ('q', value2.encode('utf-8')),
-        ]
-        self.assertEqual(req['query_params'], query_params)
+
+        expected_data = {
+            'target': 'en',
+            'q': [value1, value2],
+            'cid': (),
+            'source': None,
+            'model': None,
+            'format': None,
+        }
+        self.assertEqual(req['data'], expected_data)
 
     def test_translate_explicit(self):
         client = self._make_one(_http=object())
@@ -352,15 +364,16 @@ def test_translate_explicit(self):
         req = conn._requested[0]
         self.assertEqual(req['method'], 'POST')
         self.assertEqual(req['path'], '')
-        query_params = [
-            ('target', target_language),
-            ('q', value.encode('utf-8')),
-            ('cid', cid),
-            ('format', format_),
-            ('source', source_language),
-            ('model', model),
-        ]
-        self.assertEqual(req['query_params'], query_params)
+
+        expected_data = {
+            'target': target_language,
+            'q': [value],
+            'cid': [cid],
+            'source': source_language,
+            'model': model,
+            'format': format_,
+        }
+        self.assertEqual(req['data'], expected_data)
 
 
 class _Connection(object):