Yelp/mrjob — Issue #337: allow hadoop mode use HDFS on non-default Hadoop cluster
Author: davidmarin
Created: 2012-01-13 23:48:46+00:00
State: open
Labels: Feature

The `hadoop` binary takes an `-fs` option which allows you to pass in a URI to a non-default instance of HDFS. Currently we don't provide a way to get at this functionality.

Probably the cleanest way to get at this functionality is just to pass the `fs.default.name` jobconf variable in if it's not set to `localhost`.

Alternatively, we could have a separate `hadoop_fs_bin` option, but that seems confusing.