bwhite/hadoopy — Issue #38: Unrecognized option: -io
Author: tylere
Created: 2011-11-13 22:58:03+00:00
State: open

While running hadoopy with the current stable version of hadoop (hadoop-0.20.203.0rc1) the following error is triggered. What version of hadoop is hadoopy compatible with?

$ python -c "import hadoopy; out = hadoopy.launch('gutenberg/20417.txt.utf8', '/tmp/gutenberg/out/', 'wc.py')"
/\----------Hadoop Output----------/\
hadoopy: Running[hadoop jar /usr/local/hadoop-0.20.203.0/contrib/streaming/hadoop-streaming-0.20.203.0.jar -output /tmp/gutenberg/out/ -input gutenberg/20417.txt.utf8 -mapper "python wc.py pipe map" -reducer "python wc.py pipe reduce" -combiner "python wc.py pipe combine" -file wc.py -jobconf mapred.job.name=wc -io typedbytes -outputformat org.apache.hadoop.mapred.SequenceFileOutputFormat -inputformat AutoInputFormat]
log4j:ERROR Could not find value for key log4j.appender.NullAppender
log4j:ERROR Could not instantiate appender named "NullAppender".
11/11/13 13:49:43 ERROR streaming.StreamJob: Unrecognized option: -io