alexott/clojure-hadoop — pull request #11 by Roxxi (2013-03-10 06:20:05+00:00)
Title: Specifying key-value pairs for the JobConf via the commandline

With this patch, a user can specify key-value pairs
via the commandline to be configured in the JobConf.
Parameters can be specified by placing an 'X' infront
of the key, followed by a value that will be treated
as a String in the configuration.

For example the -X parameters below:
java -cp examples.jar clojure_hadoop.job \
-job clojure-hadoop.examples.wordcount6/job \
-input README.txt -output out6 \
-Xmy.custom.repeat.string hello \
-Xmy.custom.multiplier 11

will be set as key-value pairs in the JobConf such that
jobconf.getConfiguration().get("my.custom.repeat.string")
=> "hello"
jobconf.getConfiguration().get("my.custom.multiplier")
=> "11"

wordcount6.clj provides an written example of such usage, and
also outlines the use of the :params param that can be specified
in the job mapping which, to my knowledge, was previously undocumented.

I've also verified that in the case where :params are specified, and
conflicting key-value pairs are specified over the commandline, the
commandline will override the values specified in :params, effectively
allowing the programmer to specify default values for their application.