spotify/luigi — pull request #492 by liquid-sky (2014-10-10 14:07:14+00:00)
Title: Modify HDFS to ignore Hadoop Native Libraries warning

By default 64bit Hadoop binaries come with 32bit Native Libraries, which means that hadoop cli tools will add a header to the output:

```
14/10/10 13:38:51 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
OpenJDK 64-Bit Server VM warning: You have loaded library /usr/local/hadoop-2.3.0/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c ', or link it with '-z noexecstack'.
```

For example:

```
$ hadoop fs -ls /tmp
OpenJDK 64-Bit Server VM warning: You have loaded library /usr/local/hadoop-2.3.0/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c ', or link it with '-z noexecstack'.
14/10/10 14:06:30 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Found 2 items
drwx------ - hduser supergroup 0 2014-09-05 17:36 /tmp/hadoop-yarn
drwxrwxrwt - hduser supergroup 0 2014-09-10 14:42 /tmp/logs
```

The first line will be directed to `stderr` by subprocess and the latter two will be stored in `stdout`.