Repository: spotify/luigi

Issue:
TypeError: 'str' does not support the buffer interface in luigi.contrib.hive
Hi,

I'm running luigi task with python3.4 and trying call print(HiveTableTarget(table = "tbl", database = "db").exists()).
I'm getting error the following stacktrace:

```
Traceback (most recent call last):
  File "/usr/local/lib/python3.4/dist-packages/luigi/worker.py", line 137, in run
    new_deps = self._run_get_new_deps()
  File "/usr/local/lib/python3.4/dist-packages/luigi/worker.py", line 88, in _run_get_new_deps
    task_gen = self.task.run()
  File "hive.py", line 10, in run
    print(str(target.exists()))
  File "/usr/local/lib/python3.4/dist-packages/luigi/contrib/hive.py", line 376, in exists
    return self.client.table_exists(self.table, self.database)
  File "/usr/local/lib/python3.4/dist-packages/luigi/contrib/hive.py", line 141, in table_exists
    return stdout and table.lower() in stdout
TypeError: 'str' does not support the buffer interface
```

I changed the last line in the stacktrace into 

```
return stdout and table.lower() in str(stdout)
```

and it works now.

Is this a bug or do I use wrong python version or something else?

Thanks



Discussion:
It's probably just that our python 3 users haven't tried out Hive yet. You was the first one. Please add a test case and fix the bug in a PR. :)

Hi,
This issue still exists. @red1ynx fix seems to be ok. @Tarrasch what do you need to include this fix in the next release?
ping @Tarrasch @red1ynx  can we create a PR for this. Fix seems trivial here. 
@pmerienne You can create at PR with this fix included. See our CONTRIBUTING.md file. Good luck! :)

Resolving patch:
diff --git a/luigi/contrib/hive.py b/luigi/contrib/hive.py
--- a/luigi/contrib/hive.py
+++ b/luigi/contrib/hive.py
@@ -68,7 +68,7 @@ def run_hive(args, check_return_code=True):
     if check_return_code and p.returncode != 0:
         raise HiveCommandError("Hive command: {0} failed with error code: {1}".format(" ".join(cmd), p.returncode),
                                stdout, stderr)
-    return stdout
+    return stdout.decode('utf-8')
 
 
 def run_hive_cmd(hivecmd, check_return_code=True):


Test patch:
diff --git a/test/contrib/hive_test.py b/test/contrib/hive_test.py
--- a/test/contrib/hive_test.py
+++ b/test/contrib/hive_test.py
@@ -32,7 +32,7 @@ class HiveTest(unittest.TestCase):
     def mock_hive_cmd(self, args, check_return=True):
         self.last_hive_cmd = args
         self.count += 1
-        return "statement{0}".format(self.count)
+        return six.u("statement{0}".format(self.count))
 
     def setUp(self):
         self.run_hive_cmd_saved = luigi.contrib.hive.run_hive
@@ -262,7 +262,7 @@ def test_run_hive_command(self, popen):
         # I'm testing this again to check the return codes
         # I didn't want to tear up all the existing tests to change how run_hive is mocked
         comm = mock.Mock(name='communicate_mock')
-        comm.return_value = "some return stuff", ""
+        comm.return_value = six.b("some return stuff"), ""
 
         preturn = mock.Mock(name='open_mock')
         preturn.returncode = 0
@@ -275,7 +275,7 @@ def test_run_hive_command(self, popen):
         preturn.returncode = 17
         self.assertRaises(luigi.contrib.hive.HiveCommandError, luigi.contrib.hive.run_hive, ["blah", "blah"])
 
-        comm.return_value = "", "some stderr stuff"
+        comm.return_value = six.b(""), "some stderr stuff"
         returned = luigi.contrib.hive.run_hive(["blah", "blah"], False)
         self.assertEqual("", returned)